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EXECUTIVE  SUMMARY 


This  report  documents  the  Lincoln  Laboratory  evaluation  of  the  Traffic  Alert  and  Collision 
Avoidance  System  n  (TCAS II)  logic  version  7. 

BACKGROUND 

TCAS  n  is  an  airborne  collision  avoidance  system  required  since  December  1993  by  the  Federal 
Aviation  Administration  (FAA)  on  all  air  carrier  aircraft  with  more  than  30  seats  operating  in  the 
United  States  airspace.  The  FAA  mandated  the  current  TCAS  11  logic  version  6.04a  by 
December  1994  in  order  to  correct  a  potential  safety  problem  in  earlier  versions  of  the  TCAS  n 
logic.  Version  7,  also  known  as  Change  7,  is  a  major  revision  to  the  6.04a  logic.  It  provides 
enhancements  to  all  major  TCAS  areas  (surveillance.  Collision  Avoidance  System  (CAS)  logic, 
and  displays/aurals)  and  is  essentially  equivalent  to  the  Airborne  Collision  Avoidance  System  n 
(ACAS  n),  the  international  version  that  has  been  mandated  world-wide. 

Historically,  Lincoln  Laboratory  has  been  responsible  for  the  surveillance  area  of  TCAS.  In 
addition,  however,  Lincoln  Laboratory  has  been  involved  in  the  testing  of  the  MITRE-developed 
"CAS  logic,"  the  logic  that  provides  for  threat  declaration  and  resolution.  Lincoln  Laboratory 
has  previously  worked  in  collaboration  with  the  William  J.  Hughes  Technical  Center  (WJHTC), 
formerly  known  as  the  FAA  Technical  Center  (FAATC),  to  assess  CAS  logic  performance. 
Based  on  past  success,  both  organizations  were  tasked  to  evaluate  the  Change  7  CAS  logic.  This 
report  covers  only  the  Lincoln  Laboratory  evaluation. 

This  work  was  made  necessary  by  the  difficulties  of  designing  and  validating  improvements  to 
the  TCAS  logic.  The  logic  itself  is  complicated  and  can  behave  quite  differently  from  encounter 
to  encounter.  In  many  cases  a  fix  that  resolves  problems  with  a  particular  type  of  encounter  will 
result  in  poorer  performance  in  other  types  of  encounters.  Complicating  this  situation  is  the  fact 
that  there  are  some  encounters  that  a  CAS  should  not  be  expected  to  resolve  (e.g.  encounters  in 
which  the  intruder  maneuvers  sharply  in  a  way  that  is  contrary  to  the  CAS  advisory). 

It  is  literally  possible  to  define  an  uncountable  number  of  different  encounter  parameter  sets, 
each  of  which  has  the  potential  of  revealing  different  aspects  of  CAS  logic  performance. 
Because  of  the  huge  number  of  possible  encounter  scenarios,  it  is  impossible  to  test  all  possible 
encounter  scenarios.  Thus,  a  carefully  designed  set  of  procedures  is  required  to  sample  the 
encounter  space,  characterize  the  types  of  performance  problems  that  exist,  and  ensure  that 
proposed  design  improvements  are  beneficial  in  a  universal  sense.  This  report  describes  the  set 
of  procedures  and  associated  software  that  were  developed  and  exercised  to  support  development 
of  TCAS  Change  7  logic. 


METHODOLOGY 

The  data  analyzed  by  Lincoln  Laboratory  were  generated  by  the  WJHTC  simulation  program 
known  as  the  Fast  Time  Encounter  Generator  (FTEG).  Approximately  two  million  simulated 
pairwise  encounters  were  produced  by  the  WJHTC.  The  aircraft  parameters  (e.g.,  planned 
vertical  separation  at  closest  point  of  approach,  vertical  speed,  and  acceleration  rates)  used  in 
these  encoimters  were  designed  to  include  and  extend  somewhat  beyond  the  typical  values  seen 
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in  the  airspace.  The  aircraft  maneuvers  were  timed  to  generate  worst  case  situations  for  TCAS  in 
order  to  be  able  to  test  the  performance  limits  of  the  system. 

During  previous  logic  evaluation  work,  Lincoln  Laboratory  developed  five  analysis  programs. 
These  five  programs  were  modified  to  operate  with  the  Change  7  simulated  encounter  data.  A 
new  set  of  programs  was  designed  to  help  in  the  analysis  of  TCAS— TCAS  reversed  sense 
encounters.  All  of  the  analysis  programs  are  described  in  the  report  and  sample  results  are  given. 
The  performance  metric  used  in  all  of  the  analysis  programs  is  the  vertical  separation  between 
the  two  simulation  aircraft  at  closest  point  of  approach  (CPA).  Encounters  were  categorized  as 
unacceptable  if  the  vertical  separation  at  CPA  was  100  feet  or  less,  defined  as  a  near  mid-air 
collision  (NMAC).  During  the  logic  analysis  TCAS  encounters  were  compared  to  the  "planned 
encounter,"  i.e.,  what  would  have  happened  if  TCAS  were  not  present.  This  planned  encounter 
gives  a  reference  point  for  determining  if  TCAS  failed  to  resolve  an  existing  NMAC,  or  if  TCAS 
induced  (caused)  an  NMAC. 

During  the  6.04A  logic  evaluation  Lincoln  Laboratory  identified  639  TCAS-TCAS  simulated 
encounters  in  which  both  pilots  responded  properly  and  yet  the  encounter  resulted  in  an  NMAC. 
These  639  encounters  were  grouped  into  30  distinct  categories  based  on  encounter  geometry  and 
resolution  advisory  generated.  A  “representative  NMAC”  was  chosen  from  each  of  the  30 
categories  These  representative  NMACs  were  studied  as  a  part  of  the  Change  7  logic  evaluation. 


The  evaluation  of  the  Change  7  logic  began  with  an  interim  release  of  Change  7.  This  allowed  a 
"dry  run"  to  modify  our  existing  analysis  programs  so  they  would  work  with  the  revised 
simulation  data  formats,  and  time  to  design  and  implement  those  programs  specific  to  the 
Change  7  logic  evaluation.  The  evaluation  began  in  earnest  with  Version  7  Mod  10  and  fimshed 
with  Version  7  Mod  1 1,  as  described  below. 

EVALUATION  GOALS 

The  analysis  of  the  Change  7  CAS  logic  was  designed  to  answer  four  questions.  First,  "did  the 
new  Change  7  reversal  logic  perform  properly"?  Second,  "are  there  any  new  areas  of  concern 
with  the  Change  7  logic"?  Third,  "does  the  Change  7  logic  improve  the  performance  of  TCAS  E 
in  the  30  Representative  NMAC  encounters  identified  in  the  6.04a  logic  evaluation"?  And 
finally,  "what  are  the  performance  limits  of  the  Change  7  logic"? 

RESULTS 

During  the  Version  7  Mod  10  evaluation  process  using  the  programs  designed  to  study 
TCAS-TCAS  reversals,  Lincoln  Laboratory  very  quickly  identified  the  presence  of  multiple 
TCAS-TCAS  sense  reversals  most  of  which  ended  in  an  NMAC.  Upon  further  study,  the 
performance  of  even  single  reversals  was  questionable.  At  this  point,  further  analysis  of 
Version  7  Mod  10  was  suspended  while  the  TCAS  community  researched  Ae  cause  of  the  poor 
reversal  performance  and  searched  for  possible  solutions.  The  result  of  Ais  process  was  a  new 
version  of  the  Change  7  logic  (Version  7  Mod  11).  The  new  logic  has  a  limit  of  one 
TCAS-TCAS  reversal  per  encounter  and  some  other  improvements  to  the  reversal  logic.  This 
new  logic  also  shows  significant  improvement  in  TCAS-TCAS  reversals  performance  when 
both  pilots  followed  their  TCAS  commands. 
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The  following  results  refer  to  the  Version  7  Mod  1 1  logic.  To  answer  the  first  question  "did  the 
new  Change  7  reversal  logic  perform  properly"  programs  were  written  to  specifically  examine 
reversed  RAs.  For  this  logic  version  there  are  no  multiple  sense  reversals  in  TCAS-TCAS  or 
TCAS-imequipped  encounters.  The  sense  reversals  that  do  occur  with  the  Version  7  Mod  1 1 
logic  are  effective  i.e.,  they  are  more  likely  to  result  in  larger  vertical  separations  than  with 
Version  6.04a. 

One  of  the  most  compelling  reasons  for  allowing  TCAS-TCAS  reversals  was  to  protect  a  pilot 
that  is  following  a  TCAS  command  against  a  pilot  that  is  not  responding  (either  ignoring  or 
maneuvering  contrary)  to  a  command.  A  set  of  simulated  encounter  data  from  WJHTC,  in  which 
one  pilot  in  each  encounter  did  not  respond  to  the  TCAS  Resolution  Advisory  (RA),  was 
analyzed  to  assess  the  logic’s  performance  in  these  scenarios.  The  results  for  the  non-responding 
pilot  encounters  with  TCAS-TCAS  reversals  were  very  good.  In  the  encounters  most 
representative  of  real  mrspace  (planned  non-crossing)  the  new  logic  greatly  reduced  the  number 
of  induced  NMACs. 

To  answer  the  second  question  "are  there  any  new  areas  of  concern  with  the  Change  7  logic"  an 
overall  evaluation  of  the  logic  was  performed  using  the  remaining  five  Lincoln  Laboratory 
analysis  tools.  Overall  the  results  were  good.  NMAC  counts  were  tabulated  by  encounter  class 
(collections  of  pairwise  aircraft  encounters  related  by  geometry,  aircraft  vertical  rates  and 
accelerations,  see  Section  1.1)  and  logic  version.  In  every  case  the  Change  7  100-foot  and 
Change  7  25-foot  logic  had  either  fewer  NMACs  than  6.04a,  or  in  a  few  cases  the  same  number 
of  NMACs  as  6.04a.  Change  7  reduced  the  overall  number  of  crossing  RAs  observed,  compared 
to  6.04a.  Of  the  crossing  RAs  that  remain  with  Change  7  a  smaller  percent  result  in  NMACs, 
compared  to  the  percent  of  crossing  RAs  that  lead  to  NMACs  for  6.04a. 

Third,  for  the  Representative  NMAC  encounters  from  the  6.04a  logic  evaluation.  Change  7  with 
the  improved  100-foot  tracker  resolved  28  out  of  the  30  encounters.  The  Change  7  logic  with  the 
new  25-foot  tracker  resolved  27  out  of  the  30  encounters.  This  represents  a  significant 
improvement  over  the  6.04a  logic’s  inability  to  resolve  any  of  the  30  encounters. 

Fourth,  to  understand  the  limits  of  the  Change  7  logic,  every  NMAC  generated  by  either  the 
Change  7  100-foot  or  Change  7  25-foot  logic  was  plotted.  These  hundreds  of  plots  were  divided 
into  groups  of  similar  geometry  and  failure  mechanism.  There  were  three  groups  where  the 
Change  7  logic  failed  to  resolve  a  situation  that  would  have  been  an  NMAC  without  TCAS 
(unresolved  NMACs).  There  were  fourteen  groups  where  Change  7  caused  an  NMAC  where 
there  was  originally  250  feet  or  more  of  vertical  separation  (induced  NMACs).  Remembering 
that  the  WJHTC  simulation  is  designed  to  stress  the  limits  of  the  logic,  these  groups  of  failures 
were  studied  to  determine  if  they  were  likely  to  occur  in  the  airspace.  In  addition  there  are  some 
types  of  encounters  that  TCAS  cannot  reasonably  be  expected  to  resolve  (for  example,  a  high 
rate  climb  by  the  intruder  aircraft  20  seconds  prior  to  closest  point  of  approach).  All  of  the 
seventeen  groups  of  failures  fell  into  the  "unlikely  to  occur"  or  "impossible  to  fix"  categories. 
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1.  INTRODUCTION 


1.1  BACKGROUND 

The  Traffic  Alert  and  Collision  Avoidance  System  (TCAS)  is  an  airborne  collision  avoidance 
system,  required  since  30  December  1993  by  the  FAA  on  all  air  carrier  aircraft  with  more  than 
30  passenger  seats  operating  in  U.S.  airspace.  TCAS  works  by  actively  interrogating  other 
nearby  transponder-equipped  aircraft  and  tracking  the  transponder  replies.  For  each  aircraft, 
TCAS  computes  a  tau  value,  or  time  to  closest  approach.  When  this  value  drops  below  a 
specified  threshold,  typically  25-30  seconds,  TCAS  issues  a  vertical  command,  or  resolution 
advisory,  to  the  pilot. 

There  are  two  levels  of  TCAS.  TCAS  n  is  described  above  and  is  the  only  level  discussed  in  this 
report.  TCAS  I  is  intended  for  aircraft  with  10-30  seats  and  has  lesser  capability;  i.e.,  TCAS  I 
displays  only  traffic  advisories  (position  information)  to  the  pilot,  not  resolution  advisories. 

In  order  to  make  the  operation  of  TCAS  more  compatible  with  the  existing  air  traffic  conftol 
system  as  well  as  to  correct  a  potential  safety  problem  with  unnecessary  crossing  resolution 
advisories,  all  TCAS-equipped  aircraft  were  required  to  install  a  new  logic  version,  known  as 
version  6.04a,  by  30  December  1994. 

Since  the  introduction  of  version  6.04a,  work  has  continued  in  both  the  national  and  international 
standards  communities  to  monitor  TCAS  operation  and  propose  changes  that  would  either 
enhance  current  performance  or  correct  problems  found.  The  result  of  this  work  is  TCAS 
version  7  (or  “Change  7”),  a  substantial  revision  of  6.04a,  consisting  of  more  than  300  separately 
defined  changes  affecting  all  major  TCAS  areas.  AC  AS  n,  the  internationally-defined  collision 
avoidance  system  that  has  been  mandated  world-wide,  is  essentially  equivalent  to  TCAS 
Change  7. 

One  of  the  key  differences  between  versions  6.04a  and  7  is  that  in  TCAS— TCAS  coordinated 
encounters,  the  Change  7  logic  allows  a  TCAS  to  reverse  its  coordinated  Resolution  Advisory 
(RA)  sense  if  the  encounter  geometry  indicates  that  the  situation  is  being  degraded.  With  version 
6.04a,  once  the  coordination  had  taken  place,  no  reversals  were  permitted.  Because  Lincoln 
Laboratory  had  been  responsible  for  development  of  the  TCAS-TCAS  coordination  logic, 
Lincoln  Laboratory  was  asked  to  take  an  active  role  in  testing  MITRE’s  TCAS— TCAS  geometric 
reversal  logic.  Lincoln  Laboratory  had  previously  teamed  with  the  FAA’s  William  J.  Hughes 
Technical  Center  (WJHTC)  to  perform  an  overall  evaluation  of  the  6.04a  CAS  logic.  Now 
Lincoln  Laboratory  undertook  a  similar  evaluation  of  the  Change  7  CAS  logic,  with  particular 
attention  to  the  new  TCAS-TCAS  reversal  logic.  The  Lincoln  Laboratory  evaluation  is  the 
subject  of  this  report. 

1.1.1  Logic  Versions 

The  original  logic  mandated  in  1993  was  referred  to  as  version  6.02.  Version  6.04,  a  non- 
mandated  version,  was  made  available  in  late  1992  and  was  implemented  by  a  few  of  the  airlines 
in  order  to  make  the  TCAS  logic  more  compatible  with  the  air  traffic  control  system. 
Version  6.04  reduced  the  number  of  nuisance  advisories  primarily  by  reducing  the  protection 
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volume  about  the  TCAS  aircraft  and  by  raising  the  altitude  threshold  above  which  advisories 
would  be  issued. 

Shortly  after  the  introduction  of  version  6.04,  a  potential  safety  problem  known  as  the  “Seattle 
encounter”  was  discovered  in  both  versions  6.02  and  6.04.  Version  6.04a  was  developed  to  fix 
this  problem.  Version  6.04a  was  mandated  in  all  TCAS  installations  by  30  December  1994. 

Change  7  is  a  major  revision  to  the  6.04a  logic,  and  has  been  in  development  since  1994. 
Modifications  to  the  logic  were  made  in  response  to  change  requests  (CRFs)  and  trouble  reports 
(PTRs)  submitted  by  the  TCAS  community.  Among  the  changes  were  upgrades  to  the  vertical 
tracker  logic.  The  100-foot  vertical  tracker  was  improved  over  the  tracker  used  in  6.04a.  A  new 
tracker  using  25-foot  intruder  altimetry  data  was  implemented.  These  two  different  trackers  give 
rise  to  two  different  Change  7  logic  versions  studied. 

The  RA  display  deferral  logic  was  eliminated  for  Change  7.  Also,  the  ability  for  TCAS  to 
reverse  sense  against  a  TCAS  equipped  threat  was  added.  Previously  sense  reversals  were  only 
allowed  during  the  coordination  process  or  against  unequipped  intruders.  In  addition  to  the 
changes  mentioned  above,  a  horizontal  miss  distance  filter  was  implemented  to  reduce  the 
number  of  RAs  posted  when  there  is  adequate  horizontal  separation.  Also,  the  multi-aircraft 
logic  was  redesigned.  Lincoln  Laboratory  did  not  study  these  two  enhancements  to  the  logic 
because  the  simulation  data  from  WJHTC  did  not  provide  horizontal  position  information  or 
multi-aircraft  encounters. 

In  the  Change  7  logic  evaluation  described  in  this  report,  three  logic  versions  are  exarnined  - 
6.04a,  Change  7  using  the  100-foot  vertical  tracker  (Change  7-100),  and  Change  7  using  the 
25-foot  vertical  tracker  (Change  7-25).  This  was  due  to  the  fact  that  all  three  versions  will  be 
operated  simultaneously  in  the  airspace  for  some  period  of  time,  and  it  is  necessary  to  examine 
the  interactions  between  versions. 

During  the  Change  7  logic  evaluation  there  were  many  iterations  of  the  Change  7  logic.  In  this 
report  there  are  references  to  Interim  Release  10  Mod  1,  Version  7  Mod  10,  and  Versiori  7 
Mod  11.  Interim  Release  10  Mod  1  was  the  first  release  of  the  Change  7  logic  with 
TCAS-TCAS  reversals  that  was  examined  in  depth  at  Lincoln  Laboratory.  Version  7  Mod  10  is 
the  Change  7  logic  that  was  approved  by  SC-147  in  May  1997.  Version  7  Mod  1 1  is  a  revision 
of  the  logic  designed  to  improve  the  TCAS-TCAS  reversals  performance.  Version  7  Mod  11 
was  accepted  by  SC-147  in  November  1997  and  was  then  included  in  the  international  standards 

forACAS  n. 

1.1.2  TCAS  Development  and  Testing 

During  the  development  of  TCAS,  MITRE  has  been  responsible  for  development  of  the  CAS 
logic,  i.e.,  the  algorithms  that  perform  threat  detection  and  maneuver  selection.  Lincoln 
Laboratory  has  been  responsible  for  development  of  the  surveillance  logic,  i.e.,  the  algorithms 
for  maintaining  surveillance  on  other  aircraft,  and  the  coordination  logic,  i.e.,  the  algorithms  that 
ensure  complimentary  maneuvers  between  two  aircraft  in  an  encounter.  It  is  the  CAS  logic  that 
is  the  subject  of  this  evaluation. 

Testing  of  the  CAS  logic  is  done  by  means  of  software  simulation  of  large  numbers  of  aircraft 
encounters.  As  the  principal  developer  of  the  CAS  logic,  MITRE  has  been  responsible  for  the 
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majority  of  the  CAS  logic  testing,  especially  as  it  relates  to  assessing  the  operational  impact  and 
safety  of  different  logic  versions.  WJHTC  became  involved  in  the  CAS  logic  testing  in  order  to 
provide  an  independent  check  of  performance  and  to  provide  an  assessment  of  the  strengths  and 
weaknesses  of  the  logic.  In  1991,  Lincoln  Laboratory  was  tasked  to  work  with  WJHTC  to  help 
organize  and  analyze  the  large  amount  of  data  produced  by  the  WJHTC  simulation.  The  Lincoln 
Laboratory  analysis  tools  proved  to  be  an  excellent  predictor  of  logic  problems  and  have  been 
used  to  evaluate  several  versions  of  the  CAS  logic. 

The  WJHTC’s  TCAS  simulation  program,  referred  to  as  the  Fast  Time  Encounter  Generator  (or 
FTEG)  was  described  in  detail  in  ATC-240,  Section  2.  All  encounters  run  through  this 
simulation  belong  to  one  of  twenty  encounter  classes  based  on,  but  not  limited  to  aircraft  tracks 
recorded  at  ARTS  sites  throughout  the  United  States  before  TCAS  was  available.  These 
encounter  classes  are  shown  in  Figure  1-1.  The  higher  numbered  classes  (10-19)  contain 
encounters  where  the  aircraft  do  not  cross  in  altitude.  The  lower  numbered  classes  (0-9)  contain 
encounters  where  the  two  aircraft  cross  in  altitude.  In  Figure  1-1,  class  0  appears  to  be  two  level 
aircraft,  however  there  can  be  vertical  rates  of  up  to  400  fpm  in  class  0. 


(Line  segments  with  arrows  represent  aircraft  vertical  profiles.) 

Figure  1-1.  Encounter  classes.  (0-9  planned  crossing,  10-19  planned  non-crossing) 


The  WJHTC  provided  Lincoln  Laboratory  vtith  their  simulation  source  code,  simulation  input 
files,  and  two  forms  of  output  files.  The  first  form  of  output  file,  known  as  the  Encounter 
Recorded  Data  (ERD),  contains  a  condensed  description  of  each  encounter  run  in  the  simulation. 
The  second  form  of  output  file,  known  as  the  Lincoln  Laboratory  Parameter  file  (LLP)  is  derived 
from  the  ERD  files  and  contains  a  very  short  record  for  each  encounter  that  resulted  in  an 
NMAC. 

In  the  Change  7  evaluation,  because  of  the  large  number  of  encounters  defined,  there  were  two 
separate  FTEG  data  collection/analysis  efforts.  See  Appendix  B  for  a  breakdown  of  the  numbers 
of  encounters  run  in  each  table.  The  two  tables  below  show  the  combinations  of  logic  version 
and  pilot  response  used  in  each  of  the  two  efforts.  An  x  in  a  cell  means  that  particular 
equipage/response  combination  was  run.  For  example,  in  Table  1-1,  an  x  in  the  6.04a  row  and 
Mode  C  column  means  that  some  number  of  encounters  were  run  in  which  aircraft  1  was 
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equipped  with  TCAS  version  6.04a  and  aircraft  2  was  Mode  C-equipped.  For  all  encounters  in 
Table  1-1,  both  TCAS  pilots  responded  properly  to  the  TCAS  advisory.  The  full  Lincoln 
Laboratory  analysis  (as  described  in  Sections  2  and  4  of  this  report)  was  performed  on  the 
Table  1-1  dataset. 

Table  1-1.  Version  6.04a,  Change  7-100,  Change  7-25  PUot  Responding 


Aircraft  2 


Aircraft  1 

ModeC 

6.04a 

C7-100 

C7-25 

ModeC 

X 

X 

X 

6.04a 

X 

X 

X 

X 

C7-100 

X 

X 

X 

X 

C7-25 

X 

X 

X 

X 

In  Table  1-2,  there  were  encounters  in  which  one  pilot  did  not  respond  to  (ignored)  the  TCAS 
advisory,  inicated  as  PNR  (pilot  non-responding)  in  the  output.  Due  to  time  constraints  the 
WJHTC  decided  to  collect  only  the  Change  7-25  non-responding  data.  These  data  were 
examined  to  determine  if  the  TCAS-TCAS  reversals  improved  the  protection  against  a  non¬ 
responding  pilot. 

Table  1-2.  Version  6.04a,  Change  7/25  Pilot  Responding/Pilot  Not  Responding  (PNR) 


Aircraft  2 


Aircraft  1 

Mode  C 

6.04a 

C7-25 

6.04a  PNR 

C7-25  PNR 

Mode  C 

X 

X 

6.04a 

X 

X 

X 

X 

X 

C7-25 

X 

X 

X 

X 

X 

6.04a  PNR 

X 

X 

C7-25  PNR 

X 

X 

1.2  GOALS  OF  THE  CHANGE  7  EVALUATION 
There  were  four  goals  of  the  Change  7  evaluation; 

1.  Study  the  performance  of  the  new  TCAS— TCAS  reversal  logic.  In  particular,  look  for 
instances  of  multiple  reversals  and  cases  in  which  the  reversal  results  degraded  aircraft 
separation. 

2.  Do  a  general  evaluation  of  the  Change  7  logic,  using  all  of  the  Lincoln  Laboratory 
analysis  tools,  to  detect  and  explain  any  "areas  of  concern."  This  effort  primarily 
checks  for  areas  in  which  the  Change  7  performance  is  worse  than  the  baseline  6.04a 
performance. 
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3,  Exainine  the  performance  of  the  Change  7  logic  for  the  30  Representative  NMACs 
identified  during  the  6.04a  logic  evaluation.  Determine  if  the  expected  improvements 
occurred. 

4.  Analyze  every  Change  7  NMAC  produced  by  the  simulation  in  order  to  understand  the 
performance  limits  of  the  Change  7  logic.  For  those  NMACs  deemed  likely  to  occur  in 
the  real  airspace,  discuss  possible  courses  of  action  to  improve  the  CAS  performance. 

1.3  ORGANIZATION  OF  TfflS  REPORT 

Section  1  provides  background  on  TCAS  development  and  testing,  including  descriptions  of  the 
current  CAS  logic  version  6.04a  and  the  proposed  new  logic  known  as  TCAS  II  Change  7.  It 
also  describes  the  major  goals  of  the  Change  7  evaluation  effort. 

Section  2  describes  the  seven  programs  developed  by  Lincoln  Laboratory  to  analyze  the 
simulation  outputs.  The  operation  of  each  program  is  explained,  and  sample  outputs  are  given. 

Section  3  describes  the  analysis  of  the  TCAS-TCAS  reversals  during  the  evolution  of  the 
Change  7  logic. 

Section  4  describes  the  general  evaluation  effort. 

Section  5  discusses  the  outcome  of  running  the  30  6.04a  Representative  NMACs  with  the 
Change  7  logic. 

Section  6  describes  the  Representative  NMAC  encounters  produced  by  the  Change  7  logic. 

Section  7  gives  a  summary  of  the  Change  7  logic  and  a  historical  perspective  of  the  performance 
of  versions  6.02,  6.04, 6.04a,  and  Change  7. 
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2.  ANALYSIS  TOOLS 


A  block  diagram  showing  the  WJHTC  simulation  facility  and  the  Lincoln  Laboratory  analysis 
tools  is  given  in  Figure  2-1.  There  are  seven  main  analysis  programs,  described  in  the 
subsections  below.  Main  inputs  to  the  analysis  programs  are  the  WJHTC  Encounter  Recorded 
Data  (ERD)  files.  Lincoln  Laboratory  maintains  a  duplicate  copy  of  the  WJHTC  simulation. 
Second-by-second  data  outputs  from  Lincoln  Laboratory’s  simulation  are  used  to  generate  plots 
of  individual  encounters. 

The  performance  metric  used  in  all  of  the  analysis  programs  is  the  vertical  separation  between 
the  two  aircraft  at  point  of  closest  approach.  In  general,  encounters  are  either  acceptable  or  not 
acceptable  depending  upon  whether  or  not  the  encounter  results  in  an  NMAC,  or  near  mid-air 
collision,  defined  as  a  vertical  separation  of  <  100  feet  at  point  of  closest  approach.  (Horizontal 
separation  for  an  NMAC  is  defined  to  be  <  0.1  nmi).  Because  TCAS  does  not  attempt  to  affect 
horizontal  separation,  the  evaluation  of  TCAS  logic  performance  usually  assumes  that  a  worst 
case  (zero)  horizontal  separation  will  occur.  Thus,  avoidance  of  an  NMAC  depends  entirely  on 
achieving  sufficient  vertical  separation.) 

A  key  element  in  the  measurement  of  performance  is  the  "planned  encounter,"  i.e.,  an  encounter 
as  it  would  have  unfolded  if  TCAS  were  not  present.  The  "planned"  performance  of  each 
geometry  used  by  the  simulation  (i.e.,  the  vertical  separation  at  closest  approach)  is  determined 
by  running  the  simulation  Avith  a  TCAS  non-responding  aircraft  in  an  encounter  with  a  Mode  C 
aircraft.  This  planned  performance  is  compared  to  the  performance  of  various  TCAS  equipages 
to  determine  if  TCAS  failed  to  resolve  an  existing  NMAC  or  induced  an  NMAC  where  none  had 
previously  existed.  Note  that  according  to  international  guidelines,  for  every  100  existing 
NMACs,  the  goal  is  for  TCAS  to  be  able  to  resolve  90  NMACs  without  inducing  more  than  2 
NMACs'  Thus,  it  is  accepted  that  TCAS  will  not  be  able  to  resolve  all  NMACs,  but  there  is  a 
very  low  tolerance  for  inducing  NMACs. 

Referring  to  Figure  2-1,  there  are  seven  analysis  programs.  The  two  Reversal  Analysis  Prograins 
were  developed  specifically  for  the  Change  7  logic  evaluation.  These  programs  are  described  in 
detail  in  Section  2.1.  Four  other  programs  (Hot-Spot  Program,  Performance  Statistics  Program, 
NMAC  Characterization  Program,  and  NMAC  Analysis  Program)  were  carried  over  from  the 
6.04a  logic  analysis  and  were  described  in  detail  in  Project  Report  ATC-240,  Section  3.  The 
Matrix  Generator  program  was  carried  over  from  the  6.04a  logic  analysis,  but  WJHTC  changed 
the  number  and  definition  of  some  of  the  tables,  so  this  program  will  be  described  in  detail  in 
Section  2.2  of  this  report. 

A  significant  question  which  needed  to  be  addressed  in  the  Change  7  logic  was  the  performance 
of  the  TCAS-TCAS  reversal  logic.  Because  of  this  the  Reversal  Analysis  Programs  were  the 
first  programs  run.  Using  the  Reversal  Analysis  Programs  we  were  able  to  quickly  detect  the 
presence  of  multiple  reversals,  and  also  the  presence  of  large  numbers  of  reversed  encounters 
that  were  not  NMACs,  but  experienced  significant  loss  of  vertical  separation. 

The  Matrix  Generator  Program  was  the  next  program  run  and  provides  a  means  for  very  quickly 
and  clearly  understanding  CAS  logic  performance  (in  terms  of  NMACs)  as  a  function  of 
encounter  class  (classes  0-19)  and  equipage  pair  (6.04a  vs.  6.04a,  6.04a  vs.  Change  7-100,  etc.). 
In  cases  where  a  detailed  analysis  is  not  required  or  possible,  this  single  program  can  provide 
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extremely  useful  overview  performance  information,  both  in  absolute  terms  and  in  relative  terms 
between  the  different  logic  versions. 

The  Hot-Spot  Program  combines  related  cells  from  two  of  the  Matrix  Generator  Program  tables 
and  produces  condensed  tables.  These  tables  identify  "hot-spots"  or  areas  of  poor  performance 
in  a  given  version  of  the  TCAS  logic.  These  hot-spots  are  then  examined  in  more  detail,  first  by 
the  NMAC  Characterization  Program  and  then  by  the  NMAC  Analysis  Program,  ^e  NMAC 
Characterization  Program  identifies  particular  parameters  (vertical  rate,  acceleration,  etc.)  or 
combinations  of  parameters  associated  with  the  hot-spots.  The  NMAC  Analysis  Program  scans 
through  the  encounter  data  for  each  of  the  hot-spots,  providing  a  summary  of  key  encounter 
elements,  e.g.,  the  sequence  of  advisories  for  each  aircraft,  timing  delays  in  the  issuing  of 
advisories,  etc. 

Finally,  the  Performance  Statistics  Program  is  run  on  all  of  the  input  data.  This  program 
provides  statistics  on  the  frequency  and  performance  of  altitude  crossing  advisories. 


Figure  2-1.  Lincoln  Laboratory  Analysis  Programs. 


2.1  REVERSAL  PROGRAMS 

TCAS-TCAS  geometric  reversals  are  an  important  new  feature  in  the  Change  7  logic.  New 
software  was  developed  at  Lincoln  Laboratory  to  study  the  behavior  of  TCAS-TCAS  geometric 
reversals.  The  purpose  of  the  Reversal  Analysis  Program  is  to  provide  statistics  on  the  frequency 
and  effectiveness  of  reversed  Resolution  Advisories  (RAs).  There  are  two  variations  of  the 
Reversal  Analysis  Program.  The  first  variation  shows  statistics  for  reversed  RAs  based  on  the 
presence  or  absence  of  NMACs  as  well  as  the  number  of  sense  reversals  observed.  The  second 
variation  evaluates  all  encovmters  with  RAs,  both  reversed  and  non-reversed,  and  generates 
counts  of  separation  gains  and  separation  losses. 
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2.1.1  Multiple  Reversals 

The  first  program  variation  tabulates  information  by  equipage  pair  for  TCAS-unequipped  and 
TCAS-TCAS  reversed  encounters.  The  initial  focus  of  the  program  was  to  look  for  "fast" 
reversals,  i.e.,  those  occurring  within  three  seconds  of  the  initial  RA,  and  multiple  sense 
reversals.  Figure  2-2  shows  a  sample  output  from  Reversals  Analysis  Program  variation  one  for 
class  13,  Version  7  Mod  1 1 .  Results  for  the  full  twenty  classes  are  given  in  Appendix  C. 

As  shown  in  Figure  2-2,  for  each  equipage  there  are  fifteen  statistics  computed: 

1 .  the  percentage  of  encounters  that  produced  RAs; 

2.  the  percentage  of  encounters  that  produced  NMACs; 

3 .  the  percentage  of  NMACs  that  were  induced; 

4.  the  percentage  of  encounters  that  had  reversals; 

5 .  the  percentage  of  RAs  that  had  reversals; 

6.  the  percentage  of  reversals  that  occurred  within  three  seconds  of  the  initial  RA; 

7.  the  percentage  of  reversals  that  were  single  reversals; 

8.  the  percentage  of  single  reversals  that  were  NMACs; 

9.  the  percentage  of  single  reversal  NMACs  that  were  induced; 

10.  the  percentage  of  reversals  that  were  double  reversals; 

1 1 .  the  percentage  of  double  reversals  that  were  NMACs; 

12.  the  percentage  of  double  reversal  NMACs  that  were  induced; 

13.  the  percentage  of  reversals  that  had  at  least  tiiree  reversals; 

14.  the  percentage  of  triple  (plus)  reversals  that  were  NMACs; 

15.  the  percentage  of  triple  (plus)  reversal  NMACs  that  were  induced. 

Remember  that  class  13  is  defined  as  a  non-crossing  class  (i.e.,  the  aircraft  do  not  plan  to  cross  in 
altitude),  and  this  is  a  very  common  encounter  type  in  the  airspace  (the  fourth  highest  weight  out 
of  all  twenty  classes).  Note  that  in  Figure  2-2,  all  of  the  reversal  entries  in  the  6.04a  only  column 
are  zeros,  as  TCAS-TCAS  geometric  reversals  were  not  allowed  in  version  6.04a.  From 
Figure  2-2  we  also  observe  that  C7-100  and  C7-25  have  more  than  eight  percent  of  their  RAs 
reversing  sense.  For  C7-100  1.42%  of  the  reversed  RAs  had  NMACs,  but  C7-25  had  no  reversed 
RAs  with  NMACs.  Also  note  that  the  last  six  rows  in  Figure  2-2  and  in  all  tables  found  in 
Appendix  C  are  zero.  This  will  be  discussed  in  depth  later  in  Section  3. 

2.1.2  Separation  Differences 

The  second  program  variation  provides  an  alternative  to  the  NMAC  performance  criteria  by 
defining  a  new  measure,  the  separation  difference.  The  separation  difference  is  defined  as  the 
absolute  value  of  the  achieved  separation  (with  TCAS  involved)  minus  the  absolute  value  of  the 
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planned  separation  (i.e.,  what  would  have  occurred  without  TCAS).  As  an  example,  for  a 
planned  separation  of -500  feet  (aircraft  1  500  feet  below  aircraft  2)  and  an  achieved  separation 
of  400  feet  (aircraft  1  400  feet  above  aircraft  2)  we  get  a  separation  difference  of  400  -  500  or 
-100  feet;  a  loss  of  100  feet  in  vertical  separation.  This  value  is  computed  for  every  encounter 
that  had  an  RA.  Two  sets  of  four  tables  are  generated;  the  first  set  is  for  non-reversed  RAs  and 
the  second  set  is  for  reversed  RAs. 

MITRE  encounter  classes;  3,13  Date  processed:  8/21/97 

Based  on  FAA  Technical  Center  data  of:  AUGUST  1997 
Total  Encounters:  93312 

Total  incorrectly  labeled  RAs  :  0 

Class  13  Planned  =  NON-CROSSING 


Percent  1 

6.04A  1 
Mode  Cl 

C7-1001 
Mode  C| 

C7-25  1 
Mode  Cl 

6.04A  1 
only  1 

C7-1001 

6.04A1 

C7-25  1 
6.04AI 

C7-100I 
only  1 

C7-25  1 
only  1 

C7-25  1 
C7-100I 

RAs/  1 

runs  ! 

1 

59.291 

1 

59.17 1 

1 

58.331 

1 

62.06! 

1 

62.521 

1 

60.011 

1 

62.501 

! 

60.151 

i 

61.29  1 

NMACs/  1 

runs  1 

1 

0.761 

1 

0.371 

1 

0.221 

1 

0.101 

! 

0.051 

1 

0.021 

! 

0.071 

1 

0.001 

1 

0.041 

Induced/  I 
NMACS  1 

1 

96.77  1 

1 

100.001 

1 

100.00! 

! 

100.001 

1 

100.001 

1 

100.00! 

1 

100.001 

i 

0.001 

1 

100.00! 

Reversals/ | 
runs  1 

! 

6.891 

1 

7.87  1 

1 

9.391 

1 

0.001 

1 

2.661 

1 

2.861 

i 

5.171 

1 

5.341 

1 

5.23! 

Reversals/ 1 
RAs  1 

1 

11.621 

1 

13.301 

1 

16.091 

1 

0.001 

1 

4.251 

1 

4.761 

1 

8.271 

1 

8.881 

1 

8.541 

Fast  Rev. / | 
Reversals  I 

1 

38.431 

1 

39.251 

! 

43.341 

1 

0.001 

1 

29.951 

1 

32.191 

1 

33.651 

1 

36.241 

I 

35.131 

1  Rev.  /  1 

Reversals  | 

1 

100.001 

1 

100.001 

1 

100.001 

1 

O.OOl 

! 

100.001 

1 

100,001 

1 

100.001 

1 

100.001 

1 

100.001 

NMACS/  1 

1  Rev.  1 

1 

0.001 

1 

0.001 

1 

0.001 

1 

O.OOl 

1 

0.92{ 

1 

0.43} 

1 

1.42! 

1 

0.001 

1 

0.701 

Ind.  NMAC/I 
NMACs  ! 

1 

0.001 

1 

0.001 

1 

0.001 

1 

O.OOl 

1 

100.001 

1 

100.001 

1 

100.001 

1 

0.001 

1 

100.001 

2  Rev.  /  1 

Reversals  I 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.00! 

NMACS/  1 

2  Rev.  1 

1 

0.001 

! 

0.001 

I 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.00! 

1 

0.001 

1 

0.001 

Ind.  NMAC/I 
NMACs  1 

1 

0.001 

1 

0.001 

1 

0.00! 

1 

0.001 

1 

0.00! 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

3+  Rev.  /  1 
Reversals  1 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

NMACS/  1 

3+  Rev.  1 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

Ind.  NMAC/I 
NMACs  1 

1 

O.OOl 

1 

0.001 

1 

0.001 

1 

0.001 

1 

O.OOl 

1 

0.001 

1 

0.001 

1 

0.001 

i 

0.001 

+ 

NMACs  and  average  alt.  sep.  at  CPA  are  based  on 

simulation  truth 

Figure  2-2.  Sample  Reversal  Analysis  Program  Variation  One  Output  -  Version  7  Mod  11 
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Each  set  contains  four  tables,  showing  the  number  of:  (1)  planned  crossing  encounters  showing 
separation  gains,  (2)  planned  crossing  encounters  showing  separation  losses,  (3)  planned  non¬ 
crossing  encounters  showing  separation  gains,  and  (4)  planned  non-crossing  encounters  showing 
separation  losses.  Each  individual  table  shows  the  separation  gain  or  loss  grouped  in  250  foot 
bins  for  each  of  the  possible  planned  separations.  Results  for  the  full  twenty  classes  are  given  in 
Appendix  D.  The  separation  difference  results  for  classes  3,13  are  shown  in  Figure  2-3.  For 
Figure  2-3,  the  first  four  tables  describe  the  behavior  of  non-reversed  RAs;  notice  that  the 
"gains"  tables  are  well  populated  and  the  "losses"  tables  are  almost  all  zero  entries.  The  next 
four  tables  describe  the  behavior  of  reversed  RAs  for  Version  7  Mod  11;  here  we  see  mostly 
gains  and  a  few  losses  that  are  small  in  magnitude. 

2.2  MATRIX  GENERATOR  PROGRAM 

The  purpose  of  the  Matrix  Generator  Program  is  to  provide  an  easy-to-read  summary  of  the 
number  of  NMACs  as  a  function  of  encounter  class  and  equipage  pair  and  to  provide  a 
description  of  key  parameters  associated  with  those  NMACs.  In  particular,  the  Matrix  Generator 
Program  reads  ERD  files  and  generates  two  sets  of  outputs:  NMAC  tables  and  NMAC  parameter 
files.  The  Matrix  Generator  Program  is  unique  among  the  seven  Lincoln  Laboratory  analysis 
programs  in  that  it  was  coded  at  WJHTC  based  on  Lincoln  Laboratory  specifications.  The  other 
six  analysis  programs  were  produced  solely  by  Lincoln  Laboratory. 

2.2.1  NMAC  Tables 

There  are  eight  NMAC  tables,  or  matrix  tables,  for  each  encounter  class.  The  Matrix  Generator 
Program  generates  these  tables  according  to  the  scheme  shown  in  Figure  2-4.  First,  the  TCAS 
encounters  for  each  class  are  divided  into  two  groups:  those  whose  corresponding  planned 
encounter  resulted  in  an  NMAC  and  those  whose  corresponding  planned  encounter  did  not  result 
in  an  NMAC.  From  the  first  group  (planned  NMACs),  the  program  then  looks  at  the  vertical 
separations  produced  when  the  aircraft  in  the  encounters  are  equipped  with  TCAS.  The 
encounters  are  then  divided  into  three  subgroups: 

A:  neither  aircraft  had  a  resolution  advisory  (RA),  but  an  NMAC  resulted  (TCAS  had  a 
missed  detection); 

B:  at  least  one  aircraft  had  an  RA,  but  still  an  NMAC  resulted  (TCAS  couldnt  resolve  the 
original  bad  situation); 

C:  at  least  one  aircraft  had  an  RA,  but  there  was  no  NMAC  (TCAS  resolved  the  original 
bad  situation). 

From  the  second  group  (planned  non-NMACs),  the  program  then  looks  at  the  vertical  separations 
produced  when  the  aircraft  in  the  encounters  are  equipped  with  TCAS.  The  encounters  are  then 
divided  into  four  subgroups: 

D:  neither  aircraft  had  an  RA,  and  there  was  no  NMAC  (TCAS  correctly  did  not  perceive 
there  to  be  a  problem); 

E:  at  least  one  aircraft  had  an  RA,  and  there  was  an  NMAC  (TCAS  induced  an  NMAC); 
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F:  at  least  one  aircraft  had  an  RA,  there  was  no  NMAC,  but  there  was  a  loss  in  vertical 

separation. 

G:  at  least  one  aircraft  had  an  RA,  there  was  no  NMAC,  and  there  was  increased  vertical 
separation. 

From  the  seven  subgroups,  eight  tables  are  formed; 

Table  1  =  subgroup  A  (unresolved  NMACs  with  no  RA,  i.e.,  missed  detections) 

Table  2  =  subgroup  B  (unresolved  NMACs  with  at  least  one  RA) 

Table  3  =  Table  1  +  Table  2  =  total  number  of  unresolved  NMACs. 

Table  4  =  subgroup  E  (induced  NMACs) 

Table  5  =  subgroup  F  (reduced  separation  non-NMACs) 

Table  6  =  subgroup  G  (increased  separation  non-NMACs) 

Table  7  =  subgroup  A  +  subgroup  D  (encovmters  without  an  RA) 

Table  8  =  subgroup  C  (number  of  RAs  resolving  NMACs) 

Tables  are  labeled  based  on  the  encounter  class  and  table  number,  e.g..  Table  1.4  corresponds  to 
class  1,  induced  NMACs.  Generally,  out  of  the  five  tables,  tables  3  and  4,  unresolved  NMACs 
and  induced  NMACs,  are  used  most  frequently . 

A  sample  table.  Table  19.4,  is  shown  in  Figure  2-5.  Note  that  the  table  header  refers  to 
“simulation  truth.”  The  declaration  of  an  NMAC  uses  the  simulation's  altitude  inputs  to  the  CAS 
logic,  not  the  CAS  tracked  altitudes.  Since  simulation  truth  is  the  same  for  both  aircraft  the 
Matrix  Generator  Program  generates  the  NMAC  tables  with  the  results  of  the  FTEG  simulation 
from  only  one  aircraft  point  of  view. 

As  shown  in  Figure  2-5,  the  number  of  planned  TCAS— TCAS  encounters  is  twice  the  number  of 
planned  TCAS-Mode  C  encounters.  There  are  more  TCAS-TCAS  encounters  because  the 
Mode  S  ID  is  varied  to  test  the  CAS  air-to-air  coordination  logic.  If  both  aircraft  selert  the  same 
sense  RA,  the  low  ID  aircraft  prevails  and  the  high  ID  aircraft  must  reverse  sense.  This  is  called 
a  coordination  reversal.  Thus  each  geometry  is  run  first  with  aircraft  1  having  the  low  Mode  S  ID 
and  then  with  aircraft  2  having  the  low  Mode  S  ID. 

A  full  set  of  160  NMAC  tables  (20  classes,  8  tables  per  class)  is  given  in  Appendix  E. 

2.2.2  Parameter  Files 

The  parameter  files  provide  a  quick  summary  of  parameter  values  for  each  of  the  NMAC 
encounters.  Each  line  in  a  parameter  file  corresponds  to  one  encounter.  Since  Aere  is  too  much 
data  to  print  each  line  on  a  single  page,  the  lines  have  been  broken  up  into  three  parts. 
Appendix  F  contains  a  complete  description  of  the  parameters  in  all  three  parts,  as  well  as  a 
sample  parameter  file  printout  for  encounter  classes  7/17. 
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SEP.  DIFF.  =  ABS{ACH.  SEP.)  -  ABS(PIiAN  SEP.)  DATASET  1  CLASSES  313  CH7  VS  CH7  ONLY 
PLANNED  CROSSING  14784  ENCOUNTERS  PLANNED  NON-CROSSING  16320  ENCOUNTERS 
- ^lON  REVERSED  RAs  —  GAINS  ;  15169  LOSSES  :  10  NO  CHANGE  :  3598 


PLANNED  =  CROSSING 


RA  COUNT  =  9628  NMAC  COUNT  =  0 


GAINS 


LOSSES 


PLANNED 


GAINS 


LOSSES 


PLAN 

SEP  INO  GAINI 

>0..250  1 

>250. 

0 

FT  1 

0 

0  1 

170 

250 

FT  1 

0 

210  1 

2448 

500 

FT  1 

2306 

632  1 

48 

750 

FT  1 

106 

48  I 

14 

1000 

FT  1 

0 

0  1 

0 

PLAN 

SEP  1 

>0..250  1 

>250.  ., 

0 

FT  1 

0  1 

0 

250 

FT  1 

0  1 

0 

500 

FT  1 

0  1 

0 

750 

FT  1 

4  1 

0 

1000 

FT  1 

1 

0  1 

0 

=  NON-CROSSING 

RA  COUNT  = 

=  9149 

PLAN 

SEP  INO  GAIN 

>0. .250 

>250.  .! 

0 

FT  1 

0 

0 

52 

250 

FT  1 

0 

37 

1585 

500 

FT  1 

898 

1055 

870 

750 

FT  1 

146 

626 

4  91 

1000 

FT  I 

142 

239 

299 

>500. .750 
1987 
372 
78 
2 
0 

>500. .750 
0 
0 
0 
0 
0 


>750, .1000 
4  01 
262 
24 
0 
0 

>750. .1000 
0 
0 
0 
0 
0 


NMAC  COUNT  =  0 


PLAN  SEP 
0  FT 
250  FT 
500  FT 
750  FT 
1000  FT 


1  >0. 


.250 

0 

6 

0 

0 

0 


>250 


,  .500 
0 
0 
0 
0 
0 


>500. .750 
428 
762 
490 
226 
84 

>500. .750 
0 
0 
0 
0 
0 


>750. .1000 
32 
308 
181 
88 
4 


>750. 


.1000 

0 

0 

0 

0 

0 


>  1000 
304 
196 
16 
0 
0 

>  1000 
0 
0 
0 
0 
0 


1000 

0 

46 

54 

0 

0 

1000 

0 

0 

0 

0 

0 


-REVERSED  RAs 


GAINS 


897  LOSSES 


41  NO  CHANGE 


PLANNED  =  CROSSING 


RA  COUNT  =  82  NMAC  COUNT  =  0 


GAINS 


LOSSES 


PLAN  SEP 
0  FT 
250  FT 
500  FT 
750  FT 
1000  FT 

PLAN  SEP 
0  FT 
250  FT 
500  FT 
750  FT 
1000  FT 


NO  GAIN  I 


>0. .250 
0 
0 
0 
0 
0 


I  >0. 


.250 

0 

0 

0 

0 

0 


>250. .500 
0 
0 
0 
0 
0 

>250. ,500 
0 
0 
0 
0 
0 


>500. .750 
70 
0 
0 
0 
0 

>500. .750 
0 
0 
0 
0 
0 


>750. .1000 
12 
0 
0 
0 
0 

>750. .1000 
0 
0 
0 
0 
0 


>  1000 
0 
0 
0 
0 
0 

>  1000 
0 
0 
0 
0 
0 


PLANNED  =  NON-CROSSING 


RA  COUNT  =  856  NMAC  COUNT  =  6 


GAINS 


LOSSES 


PLAN  SEP 
0  FT 
250  FT 
500  FT 
750  FT 
1000  FT 

PLAN  SEP 
0  FT 
250  FT 
500  FT 
750  FT 
1000  FT 


NO  GAIN! 
0  I 
0  I 
0  I 
0  I 
0  I 


>0. .250 
0 

123 

104 

0 

0 


I  >0 


,  .250 
0 
31 
10 
0 
0 


>250. .500 
0 

458 

62 

0 

0 

>250. .500 
0 
0 
0 
0 
0 


>500. .750 
0 

68 

0 

0 

0 

>500. .750 
0 
0 
0 
0 
0 


>750. .1000 
0 
0 
0 
0 
0 

>750. .1000 
0 
0 
0 
0 
0 


>  1000 
0 
0 
0 
0 
0 

>  1000 
0 
0 
0 
0 
0 


Figure  2-3.  Sample  Reversal  Analysis  Program  Variation  Two  Output. 
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Table  x.5  Number  of  non-NMAC  RAs  with  reduced  separation  [F] 

Table  X.6  Number  of  non-NMAC  RAs  with  increased  separation  [G] 

Table  x.7  Number  of  encounters  without  an  RA  [A+D] _ 

Table  x.8  Number  of  RAs  resolving  NMACs  [C] _ 

X  =  encounter  class 

Figure  2-4«  Scheme  for  Generating  NMAC  Tables. 


2.3  HOT-SPOT  PROGRAM 

The  purpose  of  the  Hot-Spot  Program  is  to  check  for  areas  of  concern,  defined  as  matrix  table 
cells  for  which  logic  versions  Change  7-100  or  Change  7-25  have  more  NMACs  than  version 
6.04a.  These  areas  of  concern  are  known  as  "hot-spots". 

The  outputs  of  the  Hot-Spot  Program  are  a  compressed  form  of  the  NMAC  tables  called 
summary  NMAC  tables.  The  summary  NMAC  tables  have  the  same  table  numbers  as  the 
NMAC  tables  from  which  they  are  derived,  but  there  are  only  two  summary  NMAC  tables  per 
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encounter  clnss  (tables  x.3  and  x.4,  i.e.,  unresolved  and  induced  NMACs),  instead  of  eight,  as  in 
the  original  NMAC  tables.  In  addition,  the  numbers  in  the  summary  NMAC  tables  are  not  raw 
counts,  but  rather  percentages  (percentage  of  NMAC  encounters  out  of  the  total  number  of 
encounters  run)  to  aJlow  for  easy  recognition  of  hot-spots. 

MITRE  encounter  Class:  19  "planned  =  NON  CROSSING"  Date  processed  8/25/97 
Based  on  FAA  Technical  Center  data  of:  AUG  1997 

All  Responding  V6.04A  vs  V7  V&V  Baseline  Modi;  100  &  25-foot  trackers 
Failure:  CPA  <-  100.0  ft  based  on  simulation  truth 

Table  19.4 

Number  of  induced  Failures 
(based  on  simulation  truth) 

Normalizing  Number  =  number  of  planned  encounters  not  resulting  in  failure 
Normalizing  number  for  Mode  C  cells:  3702 

Normalizing  number  for  TCAS-TCAS  cells:  7404 

Aircraft2 


Non- 

-TCAS 

1 

V6.04a 

Ch7/100 

1 

Ch7/25 

1 

- 

A 

i 

Non-TCAS 

! 

1 

__ 

1 

! 

115 

1  48 

1 

1 

30 

1 

1 

— 

r 

c 

V6.04a 

! 

1 

35 

1 

1 

36 

1  18 

1 

1 

21 

1 

1 

r 

a 

Ch7/100 

1 

i 

22 

1 

1 

24 

i  13 

1 

! 

10 

1 

1 

— 

f 

t 

Ch7/25 

1 

1 

8 

1 

1 

5 

!  9 

1 

1 

3 

1 

1 

1 

— 

1 


Figure  2-5.  Sample  NMAC  Table. 


Table  19.4  -  Percent  of  induced  failures 

I  6.04A  i  Ch7-100|  Ch7-25  1  6.04A/  i  6.04A/  I  Ch7-25/ i 

I  only  I  ft  only  I  ft  only  I  Ch7-100  |  Ch7-25  |  Ch7-100| 


1 

TCAS-TCAS  1 

1 

0.486  1 

1 

0.176  1 

1 

0.041  ! 

! 

0.284  1 

1 

0.176  1 

1 

0.128  ! 

1 

! 

2.026  1 

1 

0.945  1 

1 

0.513  1 

! 

1 

1 

1 

-  1 

One  Mode  C  1 

—  —  1 

1 

i 

Figure  2-6.  Sample  Summary  NMAC  Table. 

The  technique  for  computing  the  summary  NMAC  tables  is  described  in  ATC-240, 
Section  3.2.1.  Figure  2-6  shows  a  sample  summary  NMAC  table,  summary  NMAC  Table  19.4. 
Summary  NMAC  tables  are  produced  by  combining  the  cells  from  NMAC  Table  19.4,  shown  in 
Figure  2-5. 

In  the  summary  NMAC  tables  hot-spots  are  indicated  by  double  asterisks  (**)  followed  by 
double  greater  than  signs  (»).  This  indicates  that  the  Change  7  performance  was  worse  than 
6.04a  and  at  least  two  percent  of  the  encounters  run  were  NMACs.  Table  cells  with  only  double 
asterisks  indicate  an  increase  in  NMACs  compared  to  6.04a,  but  an  insignificant  number  of 
NMACs  (less  than  two  percent  of  encounters  run). 

Summary  NMAC  tables  for  the  full  20  classes  are  given  in  Appendix  G. 


15 


2.4  NMAC  CHARACTERIZATION  PROGRAM 

The  purpose  of  the  NMAC  Characterization  Program  is  to  identify  particidar  parameters  (vertical 
rates,  accelerations,  etc.)  or  combinations  of  parameters  associated  with  groups  of  N^Cs. 
These  programs  were  developed  during  previous  logic  evaluations  and  are  described  in  detail  in 

ATC-240,  Section  3.3. 

There  are  two  variations  of  the  NMAC  Characterization  Program.  The  first  variation  (NlV^Cs 
as  a  function  of  parameter  values)  looks  at  the  NMACs  occurring  in  a  particular  matrix  cell  and 
determines  the  frequency  of  certain  parameters  or  parameter  combinations.  For  example, 
consider  class  19.  As  shown  in  Table  2-1  (taken  from  Appendix  A),  there  are  eight  parameters 

in  class  19. 


Table  2-1.  Class  19  Parameter  Variations 


Parameter 

Range 

Step  Size 

No.  of  Steps 

altitude  separation  @  CPA 

-1000, 1000  ft 

250 

9 

vertical  rate  1 

1000,  5000  fpm 

2000 

3 

vertical  rate  2 

-5000,  5000  fpm 

20000 

6 

vertical  acceleration  1 

-15,  -.05 

.1 

2 

vertical  acceleration  2* 

.05,  .35 

.1 

4 

time  of  acceleration  1 

25  sec 

1 

time  of  acceleration  2 

20,  30  sec 

5 

3 

Altitude  of  AC1  at  CPA 

3700, 7500  ft 

3800 

2 

•Sign  of  acceleration  is  opposite  sign  of  vertical  rate 

Now  look  at  Table  19.4  (Figure  2-5)  and  at  a  particular  cell  (3,3)  in  that  he.  Class  19 
induced  NMACs,  versions  C7-100/C7-100.  The  table  shows  that  there  were  13  NMACs  in  that 

cell. 


Figure  2-7  shows  an  output  from  the  first  NMAC  Characterization  Program  corresponding  to 
that  same  cell  Note  that  the  headings  on  the  top  right  of  the  table  correspond  to  tlm  eight 
parameters  in  Table  2-1  above.  For  each  of  the  eight  parameters  (plus  a  mnih  ’  ^  ffi),  the 

output  shows  the  frequency  with  which  particular  parameter  values  occurred  in  the  13  NMACs. 
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Figure  2-7.  Table  19.4  Cell  (3^)  NMACs  as  a  Function  of  Parameter  Values. 


For  example,  looking  at  the  column  labeled  "count,"  we  see  that  of  the  13  NMACs,  there  were  3 
in  which  aircraft  2  had  the  higher  Mode  S  ID  and  10  in  which  aircraft  1  had  the  higher  Mode  S 
TD  Likewise,  moving  down  the  column,  we  see  that  2  of  the  NMACs  had  a  planned  separation 
of -250  feet,  jmd  11  had  a  planned  separation  of -500  feet.  This  type  of  output  shows  quickly 
whether  any  parameter  value  was  especially  troublesome. 

Appendix  H  lists  NMACs  as  a  function  of  parameter  values  for  all  classes.  Tables  x.3  and  x.4 
(i.e.,  unresolved  and  induced  NMACs).  The  column  heading  65  represents  6.04a/6.04a,  the 
column  heading  71  represents  C7-100/C7-100,  and  the  column  heading  75  represents 
C7-25/C7-25. 

The  second  variation  (NMACs  as  a  function  of  logic  version)  looks  at  the  NMACs  occurring  in  a 
particular  matrix  table  and  determines  which  NMACs  are  common  to  which  versions  of  the 
logic.  For  example,  in  Table  19.4  (Figure  2-5),  the  6.04a/6.04a  cell  shows  36  NMACs,  while  the 
C7-100/C7-100  cell  shows  13  NMACs.  This  program  allows  us  to  answer  questions  such  as, 
"Are  the  13  C7-100/C7-100  NMACs  a  subset  of  the  36  6.04a/6.04a  NMACs?  Did  the  Change  7 
logic  introduce  new  NMACs  not  present  in  the  6.04a  logic?" 

Appendix  I  lists  NMACs  as  a  function  of  logic  version  for  all  classes,  tables  x.3  and  x.4  (i.e., 
unresolved  and  induced  NMACs).  There  are  three  sets  of  tables  reporting  NMACs  as  a  function 
of  logic  version.  The  first  set  compares  6.04a/6.04a  vs.  C7-100/C7-100.  The  next  set  compares 
6.04a/6.04a  vs.  C7-25/C7-25.  The  final  set  compares  C7-100/C7-100  vs.  C7-25/C7-25. 


2.5  NMAC  ANALYSIS  PROGRAM 

The  purpose  of  the  NMAC  Analysis  Program  is  to  understand  why  an  NMAC  occurred  for  one 
particular  TCAS  equipage  pair  and  not  another,  e.g.,  why  a  Change  7/Change7  encounter  had  an 
NMAC,  but  a  6.04a/6.04a  encounter  did  not.  The  program  output  is  a  set  of  encounter 
summaries,  giving  key  information  about  the  motion  of  both  aircraft,  the  CAS  logic  thresholds  in 
use,  the  specific  event  that  triggered  the  RA,  the  sequence  of  RAs,  and  the  presence  of  reversed 
RAs.  Appendix  J  contains  a  summary  of  CAS  thresholds,  layers,  and  sensitivity  levels  for 
reference. 

The  NMAC  Analysis  Program  was  developed  during  the  6.04a  logic  evaluation  and  is  described 
in  detail  in  ATC-240,  Section  3.4.  For  the  Change  7  analysis,  three  sets  of  encounter  summaries 
were  printed  together,  allowing  quick  comparison  of  the  differences  between  logic 
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versions  6.04a,  C7-100,  and  C7-25.  In  addition,  because  most  of  the  analysis  for  Change  7 
centered  on  reversed  RAs,  any  sequence  of  RAs  containing  a  sense  reversal  is  preceded  by  ** 
REV  **"  in  the  encounter  summaries.  A  sample  set  of  three  encounter  summaries  is  shown  in 
Figure  2-8  with  a  description  of  the  fields  in  Figure  2-9.  In  Figure  2-9,  the  term  RA  final 
condition  means  that  this  is  the  final  test  that  was  passed  in  the  CAS  logic  Range  and  Altitude 
tests  before  the  RA  was  issued.  Possible  values  for  RA  final  condition  are  range  tau  (TAUR,) 
vertical  tau  (TAUV),  relative  altitude  between  the  two  aircraft  (RELZ),  and  projected  vertical 
miss  distance  (PVMD).  Also  in  Figure  2-9,  ALIM  is  the  vertical  separation  that  the  CAS  logic  is 
intending  to  achieve. 


1541  6.04A  RH  VS  6.04A  RL  18  -144.58  CROSSING__ENCOUNTER 

SL  =  5  ZTHR  =  600.0  TAUR  =  25.0  TAUV  -  25.0  ALIM  -  350.0 

750.0  (-5000.0,0.0)  (0.0,-3000.0)  0.15  -0.05  -25.0  -25.0  3700.0 

A/ci:  1165122  TA  TIME  :19  ITAUR  !  POTRA  @34  (6FT)  |  DES  @45 

[XRA]  I  IDES  @47 

A/C2:  1265022  TA  TIME  :19  (TAUR  |  DDES  @43  [XRA]  I  CL  @47 

1541  C7  100  FT  RH  VS  C7  100  FT  RL  18  2.12  CROSS IN G_ENCOUNTER 

SL  =  5  ZTHR  =  600.0  TAUR  =  25.0  TAUV  =  25.0  ALIM  =  300.0 

750.0  (-5000.0,0.0)  (0.0,-3000.0)  0.15  -0.05  -25.0  -25.0  3700.0 

A/Cl:**  REV  **1171133  TA  TIME  :19  [TAUR  1  POTRA  @34  |  POTRA  @40  i  DES 

@44  [XRA]  I  CL  @54 

A/C2:**  REV  **1271033  TA  TIME  :19  [TAUR  1  POTRA  @34  (LVW)  |  POTRA  @40 

(LVW)  (  POTRA  @42  |  DDES  @43  [XRA]  I  CL  @46  1  DES  @53 

1541  C7  25  FT  RH  VS  C7  25  FT  RL  18  1373.09  NON_CROSSING_ENCOUNTER 

SL  =  5  ZTHR  =  600.0  TAUR  =  25.0  TAUV  =  25.0  ALIM  -  300.0 

750.0  (-5000.0,0.0)  (0.0,-3000.0)  0.15  -0.05  -25.0  -25.0  3700.0 

1175144  TA  TIME  :19  [TAUR  |  CL  @34  [NXRA]  I  DDES  @46 

1275044  TA  TIME  :19  | TAUR  |  DES  @34  [NXRA]  |  IDES  @41  i 

Figure  2-8.  Sample  Encounter  Summary. 


A/ Cl: 

A/C2: 

DCL  @51 
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Reit# 

1541 


ENCOUNTER  SUMMARY  FIELD  DESCRIPTIONS 

ACl  Respond  AC2  Respond  Class  Achieved 
High  Id  Low  ID  Separation 

6,04A  RH  VS  6.04A  RL  18  -144.58  CROSS ING_ENCOUNTER 


Sensitivity  Vertical  RA  Alarm  Times 

Level  Threshold  Range  Vertical 

SL  =  5  ZTHife=600.0  TAURf=25.  TAUV=25.0 


Vertical  Separation 
goal  for  CAS  logic 

ALIM=350.0 


This  row  shows  the  values  from  the  planned  encounter: 
Planned  ACl  AC2  Accel.  Accel. 

Sep.  (beg  rate,  end  rate)  ACl  AC2 

750.0  (-5000.0,0.0)  (0.0,-3000.0)  0.15  -0,05 


Time  ACl  Alt 
ACl  AC2  at  CPA 

-25.0  -25.0  3700.0 


simulation  RA  final  delay 

mode  condition 

A/Ca*.  1165122  TA  TIME  :19  ITAOR  |  POTRA  @34  (SET) 


First  CROSSING  Next  RA 

RA  etime  RA  6  time 

I  DES  645  [XRA]  I  IDES  047 


simulation 

mode 


RA  final  First  CROSSING  Next  RA 
condition  RA  etime  RA  6  time 


A/C2:  1265022  TA  TIME  ;19  |TAUR  |  DDES  @43  [XRA]  |  CL  @47 

Reit  =  reiteration  number  an  identifying  number  for  a  particular  combination  of 
The  reit  number  and  class  uniquely  identifv 


Climb  Sense  RA  Acronyms 

Descend  Sense  RA  Acronyms 

ACRONYM 

RA 

ACRONYM 

RA 

LD2 

Limit  descent  to  2000  fpm 

LC2 

Limit  climb  to  2000  fpm 

LD1 

Limit  descent  to  1000  fpm 

LC1 

Limit  climb  to  1000  fpm 

LD5 

Limit  descent  to  500  fpm 

LC5 

Limit  climb  to  500  fpm 

DDES 

Don’t  descend 

DCL 

Don’t  climb 

CL 

Climb 

DES 

Descend 

ICL 

Increase  climb 

IDES 

Increase  descend 

Figure  2-9.  Encounter  Summary  Field  Descriptions. 


A  brief  look  at  the  encounter  summaries  in  Figure  2-8,  (referring  to  the  field  descriptions  in 
Figure  2-9)  illustrates  the  kinds  of  information  quickly  provided.  First,  by  looking  the  first  line 
of  each  summary,  one  can  tell  immediately  that  version  6.04a  did  not  have  an  NMAC  (achieved 
separation  =  -144.58  feet),  C7-100  had  an  NMAC  (achieved  separation  =  2.12  feet),  and  C7-25 
did  not  have  an  NMAC  (achieved  separation  =  1373.09  feet).  Also,  by  looking  at  the  third  field 
in  the  first  line  this  encounter  is  in  class  18,  meaning  the  aircraft  did  not  intend  to  cross  in 
altitude.  From  the  last  field  in  the  first  line  one  can  tell  that  for  6.04a  and  C7-100  this  is  a 
crossing  encounter,  but  for  C7-25,  this  is  a  non-crossing  encounter. 

Using  the  fifth  line  in  the  version  6.04a  summary  we  observe  that  AC2  issued  an  RA  ("don’t 
descend"  at  time  43  seconds  (17  seconds  prior  to  CPA)  followed  by  another  RA  ("climb")  at  time 
47  seconds.  Looking  at  the  fourth  line  of  the  version  6.04a  summary  we  see  that  ACl  issued  an 
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RA  ("descend")  at  time  45  seconds  which  was  strengthened  to  "increase  descend"  at  time  47 
seconds.  These  RAs  caused  the  aircraft  to  cross  in  altitude. 


For  this  same  encounter  with  C7-100  the  initial  RAs  were  similar  to  6.04a:  AC2  issued 
("don’t  descend")  at  time  43  seconds,  then  another  RA  ("climb")  at  time  46  seconds,  Md  ACl 
issued  an  RA  ("descend")  at  time  44  seconds.  These  RAs  would  also  cause  the  aircraft  to  cross 
in  altitude.  The  difference  shown  by  the  C7-100  logic  is  that  AC2  reversed  the  sense  of  its  ^ 
("descend")  at  time  53  seconds,  which  forced  ACl  to  reverse  to  a  "climb"  at  time  54  seconds, 
causing  the  NMAC.  In  contrast,  the  C7-25  logic  issued  RAs  much  earlier,  and  was  able  to 
choose  non-crossing  commands:  ACl  issued  an  RA  ("climb")  at  time  34  seconds,  AC2  issued  an 
RA  ("descend")  at  time  34  seconds  resulting  in  more  than  adequate  separation. 


To  aid  in  the  interpretation  of  the  encounter  summaries,  encounter  plots  were  also  Produced  A 
shell  program  was  written  to  automatically  operate  the  Lincoln  Laboratory  v^ion  of  the  FTEG 
simulation.  For  a  specified  encounter,  the  shell  program  made  calls  to  FTEG  and  produced 
encounter  plots  as  well  as  intermediate  files  used  to  generate  the  encounter  summanes. 


Figure  2-10  shows  the  output  of  the  basic  version  of  the  shell  program.  Here,  second-by-second 
position  information  is  plotted  for  the  C7-100  encounter  described  in  Figime  2-9.  ACl  is  shown 
starting  from  the  left  side  of  the  page  with  ACl  time  along  the  bottom  of  the  plot.  AC2  is  shown 
starting  from  the  right  side  of  the  page  with  AC2  time  along  the  top  of  the  plot.  The  RAs  lyued 
are  superimposed  on  the  aircraft  position.  As  you  can  see  from  the  plot,  AC2  was  level  and  had 
iust  begun  to  descend  when  the  "climb"  RA  was  issued,  i.e.,  the  RA  was  opposite  to  the  intend^ 
direction  of  the  aircraft.  ACl  was  descending  and  had  just  begun  to  level  off  when  the  descend 
RA  was  issued.  Rather  than  issue  an  increase,  the  RAs  were  reversed  six  seconds  pnor  to  CPA 

causing  the  NMAC. 


The  encounter  summaries  and  encounter  plots  were  used  throughout  the  numerous  iterations  of 
the  Change  7  analysis  effort  to  quickly  understand  the  essence  of  the  encounters. 


2.6  PERFORMANCE  STATISTICS  PROGRAM 

The  purpose  of  the  Performance  Statistics  Program  is  to  provide  statistics  on  the  frequency  and 
effectiveness  of  altitude  crossing  advisories.  The  Performance  Statistics  Pro^am  was  developed 
during  the  6.04a  logic  evaluation  and  is  described  in  detail  in  ATC-240,  Section  3.5. 

Figure  2-11  shows  a  sample  output  from  the  Performance  Statistics  Program  for  class  19.  For 
each  TCAS  equipage,  there  are  eight  statistics  computed: 

1.  the  percentage  of  encounters  that  produced  RAs; 

2.  the  percentage  of  RAs  that  were  crossing  RAs; 

3  the  percentage  of  encounters  that  produced  crossing  RAs; 

4.  the  percentage  of  crossing  RAs  that  resulted  in  NMACs; 

5.  the  percentage  of  NMACs  that  were  crossing  RAs; 

6.  the  percentage  of  encounters  that  resulted  in  NMACs; 
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7.  the  average  warning  time  in  seconds  (time  of  CPA  minus  time  of  RA); 

8.  the  average  altitude  separation  at  CPA  in  feet. 

Remember  that  class  19  is  defined  as  a  non-crossing  class  (i.e.,  the  aircraft  do  not  plan  to  cross  in 
altitude)  but  as  shown  in  the  second  row,  for  version  6.04a  nearly  24%  of  the  RAs  issued  we 
crossing  RAs.  Note  also  that  in  the  fifth  row,  for  all  of  the  equipage  pairs,  100%  of  the  NMACs 

were  crossing  RAs. 

Performance  Statistics  for  the  full  20  classes  are  given  in  Appendix  K. 

ENCOUNTER  SUMMARY  -  AIRCRAFT  ALTITUDES  (File  C7IND13BEPC) 

Data  Be  Name=CL8B\fDAT:  R0T  Number- 1541 
SIM  MODE  -  1271033  (Source:  LL  FTEG  Rua  October  1997) 


Time  in  Seconds 
fie:  c7ind13b.ext 


Figure  2-10.  Encounter  Summary  -  Aircraft  Altitudes.  Data  file  name  CL8B1_N.DAT, 

REIT  Number  =  1541,  Sim  Mode  =  1271033. 


MITRE  encounter  classes:  9,19  Date  processed:  8/25/97 

Based  on  FAA  Technical  Center  data  of:  AUGUST  1997 
Total  TCAS-TCAS  runs  for  single  point  of  view  :  139927 
Total  incorrectly  labeled  RAs  :  41 

Class  19  TCAS-TCAS  Both  Responding 


1 

1 

6.04A 

only 

Ch7-100| 
ft  only  ! 

Ch7-25  1 
ft  only  1 

6.04A/  1 
Ch7-100| 

6.04A/  1 
Ch7-25  1 

Ch7-25/ 1 
Ch7-100| 

RAs/ 

runs 

1 

(%)  1 

74.29 

1 

74.15  1 

73.00  1 

1 

74.08  1 

1 

72.13  1 

1 

73.31  1 

Crossing 

RAs 

RAs/  1 
{%)  1 

23.59 

1 

18.82  1 

18.71  1 

1 

21.12  1 

1 

20.62  1 

1 

18.78  1 

Crossing 

runs 

RAs/1 
(%)  I 

17.52 

1 

13.96  1 

13.66  1 

1 

15.64  1 

! 

14.87  1 

1 

13.77  1 

Cr.  RA  NMACs/ 1 
cross  RAs  (%)  I 

2.51 

1 

1.17  1 

0.28  1 

1 

1.68  1 

1 

1.10  1 

1 

0.87  1 

Cr.  RA  NMACs/ 1 
NMACs*  (%}i 

100.00 

1 

100.00  1 

100.00  1 

i 

100.00  1 

1 

100.00  ! 

1 

100.00  1 

NMACs*/ 

runs 

1 

(%)  1 

0.44 

1 

1  0.16  1 

0.04  1 

1 

0.26  1 

1 

0.16  1 

1 

0.12  1 

Avg  warning  I 
time**  (sec) j 

20.71 

1  1 

1  21.36  1 

21.23  1 

1 

21.43  1 

1 

21.43  ! 

I 

21.39  1 

Avg  alt 
at  CPA* 

sep  i 
(ft)  1 

979.75 

1  1 

1  981.31  1 

969.19  1 

1 

986.51  1 

1 

985.32  ) 

1 

975.20  1 

*  NMACs  and  average  alt.  sep.  at  CPA  are  based  on  simulation  truth 
**  Average  warning  time  includes  negative  times  (i.e.,  RA  occurs  after  CPA) 

Figure  2-11.  Performance  Statistics  Program,  Class  19. 
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3.  TCAS-TCAS  REVERSALS 


3.1  BACKGROUND 

Change  7  of  TCAS  n  is  a  significant  redesign  of  the  CAS  logic.  Among  the  new  features  of 
Change  7  is  the  ability  of  TCAS  to  reverse  sense  against  a  TCAS-equipped  intruder.  Earlier 
versions  of  TCAS  allowed  TCAS  to  reverse  sense  against  unequipped  intruders  only.  The 
assumption  had  been  that  TCAS  pilots  would  follow  their  RAs  and  that  TCAS— TCAS  reversals 
would  not  be  necessary.  The  assumption  of  pilot  compliance  with  all  the  RAs  turned  out  to  be 
untrue.  For  Change  7  the  TCAS  community  considered  that  the  ability  to  reverse  sense  would 
provide  some  protection  against  a  non-responding  intruder,  i.e.,  when  the  pilot  ignored  an  RA  or 
maneuvered  contrary  to  the  RA.  Because  sense  reversals  represent  a  major  operational  change  to 
the  system  it  was  important  to  show  that  the  sense  reversals  provided  a  measurable  benefit. 

Lincoln  Laboratory  emphasized  the  investigation  of  the  implementation  of  TCAS-TCAS 
reversals,  specifically  multiple  sense  reversals  and  rapid  reversals  within  a  few  seconds  of  the 
initial  RA.  The  emphasis  on  multiple  reversals  came  from  the  fact  that  once  TCAS-TCAS 
geometric  reversals  were  permitted,  there  was  no  mechanism  in  place  to  prevent  aircraft  in 
certain  geometries  from  continually  reversing  sense.  The  emphasis  on  the  rapid  reversals  came 
from  the  fact  that  Change  7  eliminated  the  3-second  display  deferral  (see  Section  3.2  below).  If 
large  numbers  of  rapid  reversals  were  seen  in  the  test  results,  then  re-introduction  of  the  display 
deferral  might  have  been  considered  and/or  pilot  training  added. 

Also,  Lincoln  Laboratory  considered  it  important  to  assess  the  performance  of  reversals  in 
general,  i.e.,  did  they  increase  or  decrease  vertical  separation.  The  Reversal  Analysis  Program, 
Variation  One  was  designed  to  address  the  multiple  reversal  and  fast  reversal  issues.  The 
Reversal  Analysis  Program,  Variation  Two  deals  more  with  overall  performance  of  reversed  RA 
encounters. 

3.2  FAST  REVERSALS 

Previous  versions  of  TCAS  logic  could  defer  the  display  of  an  RA  against  a  TCAS  equipped 
threat  for  up  to  three  seconds  to  ensure  that  the  air  to  air  coordination  between  the  two  aircraft 
was  complete.  This  delay  was  considered  necessary  because  there  is  a  chance  that  both  aircraft 
could  select  the  same  sense,  and  then  one  of  the  aircraft  would  have  to  reverse  sense  to  ensure  a 
compatible  solution.  However,  during  the  3-second  delay,  the  geometry  of  the  encounter  codd 
change  significantly  and  the  deferred  RA  might  no  longer  be  the  best  solution  to  the  conflict. 
Analysis  at  MITRE  indicated  that  the  probability  of  both  aircraft  selecting  the  same  sense 
command  at  the  same  instant  is  negligible,  so  the  display  deferral  was  removed.  This  introduced 
the  possibility  that  a  pilot  could  see  an  RA  followed  immediately  by  a  reversed  RA.  This  is 
referred  to  as  a  “fast  reversal.” 

In  the  first  release  of  the  logic  studied  by  Lincoln  Laboratory  fast  reversals  were  not  observed 
because  there  was  a  10-second  delay  after  the  initial  RA  selection  before  a  reversal  could  be 
considered.  The  10-second  delay  included  five  seconds  for  the  pilot  response  delay  and  another 
five  seconds  for  tracker  lag.  However,  because  results  with  the  first  release  of  the  logic  were 
quite  poor,  later  releases  eliminated  the  10-second  delay  and  fast  reversals  were  observed.  Some 
classes  had  as  many  as  fifty  percent  of  the  reversals  occurring  within  three  seconds  of  the  initial 
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RA  The  presence  of  fast  reversals  was  coupled  with  significantly  better  overall  reversal 
performance.  The  presence  of  fast  reversals  in  the  simulated  data  was  brought  to  Ae  attention  of 
Sie  Operations  Working  Group  of  SC-147;  they  felt  fast  reversals  could  be  handled  mth  pilot 
training.  The  consensus  of  the  TCAS  community  was  that  fast  reversals  were  acceptable,  given 
the  good  performance  in  terms  of  NMAC  reduction. 

3.3  MULTIPLE  REVERSALS 

Figure  3-1  shows  the  Reversal  Analysis  Program  Variation  One  output  for  Version  7  Intenin 
Release  10  Mod  1  (V7IR10-modl)  for  Class  7.  Using  the  summary  results  from  the  Revers^ 
Analysis  Program  Variation  One,  encounters  were  identified  in  both  airCT^  reversed  RA 
sense  repeatedly  until  they  ran  out  of  time,  finally  ending  with  an  induced  NMAC.  Notice  in 
Figure  3-1,  there  is  an  encounter  that  reversed  sense  three  times.  Several  of  Ae  inultiple  sense 
reversal  encounters  were  plotted;  the  most  extreme  example  identified  by  Ae  data  in  Fi^e  3-1 
is  shown  in  Figure  3-2.  In  Figure  3-2,  the  encounter  had  ACl  6.04a  high  Mc^e  S  ID,  ^d 
AC2  with  Change  7-25  low  Mode  S  ID.  When  AC2  reversed,  the  higher-address  6.04a  aircraft 

had  to  reverse  also. 

While  the  TCAS  community  assessed  the  acceptability  of  multiple  sense  reversals,  the  study  of 
reversed  encounters  continued  at  Lincoln  Laboratory.  The  Reversal  Analysis  Program  Variation 
One  was  expanded  to  produce  a  single  record  containing  encounter  parameters  ior  every 

reversed  encounter,  not  just  those  with  NMACs.  These  data  +/V00I 

classes  then  sorted  by  planned  separation.  For  each  planned  separation  (O  ft,  +/-  250  R  +/.5Q0  ^ 
+/-  750  ft  and  +/-  1000  ft)  the  achieved  separation  was  tabulated  as  shown  in  table  i-i.  in 
Table  3-1  the  achieved  separation  columns  contain  tallies  of  encounters  with  achieved  separation 
from  0  ft  to  250  ft;  0  ft  to  -250  ft,  0  ft  to  500  ft,  0  ft  to  -500  ft,  and  so  on.  Note  that  the  0  ft  to 
500  ft  column  includes  the  encounters  in  the  0  ft  to  250  ft  column. 

Looking  at  Table  3-1  in  the  500-foot  planned  separation  row  there  were  846 
with  109  encounters  having  multiple  sense  reversals.  In  this  same  row,  there  were  258  N^Cs 
with  10  NMACs  occurring  in  encounters  with  multiple  sense  reversals.  For  this  row  we  observe 
that  503  (326  +  177)  encounters  out  of  846  ended  with  less  than  the  planed  separation^  In 
addition  we  observe  that  408  (280  +  128)  encounters  out  of  846  ended  wiA  less  than  half  of  the 
planned  separation.  Table  3-1  showed  clearly  that  most  encounters  with  reversed  ^s  even 
Siose  that  did  not  result  in  NMACs  lost  vertical  separation.  These  results  were  presented  at  the 
November  1996  TCAS  program  review.  At  that  time,  it  was  decided  that  the  Change  7  revers 

logic  required  further  work. 
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MITRE  encounter  classes:  7,17  Date  processed:  11/13/96 

Based  on  FAA  Technical  Center  data  of:  SEPT  96 
Total  Change7  runs  :  181358 

Total  incorrectly  labeled  RAs  :  82 


Class  7  Planned  =  CROSSING  Both  Responding 

I7IR10-10017IR10-100I7IR10-100I7IR10-25  |7IR10-25  17IR10-25  ! 7IR10-25 
1  ATCRBS  1  6.04A  1  only  I  ATCRBS  !  6.04A  |  only  I7IR10-100 


RAs/  1 

runs  {%) 1 

1 

99.581 

1 

99.181 

1 

99.011 

1 

99.401 

1 

99.261 

1 

99.131 

1 

99.011 

Total  NMACs/  | 
runs  ( % ) 1 

1 

9.231 

1 

1.921 

1 

1.601 

1 

6.741 

1 

1.17  1 

1 

0.831 

1 

1.04  1 

Induced  /  1 

NMACS  (%)l 

1 

73.24  1 

1 

97.701 

1 

100.00! 

1 

76.341 

{ 

100.001 

1 

100.00! 

1 

100.001 

Reversals/  1 

runs  (%) 1 

1 

5.381 

1 

0.221 

1 

0.671 

i 

6.001 

1 

0.101 

t 

0.341 

1 

0.461 

Reversals/  1 

RAs  (%)  1 

1 

5.401 

1 

0.231 

1 

0.681 

1 

6.031 

1 

0.111 

1 

0.35! 

1 

0.471 

Fast  Reverse/ | 
Reversals  (%) 1 

1 

2.53j 

1 

1.791 

1 

72.621 

1 

0.67  1 

1 

3.851 

1 

76.741 

1 

74.781 

1  Reversal/  1 
Reversals  {%) I 

1 

98.66! 

1 

69.641 

1 

89.291 

1 

100. OOj 

1 

76.92! 

1 

100.001 

1 

93.911 

NMACS/  1 

1  Rev.  {%) ! 

1 

1.061 

1 

15.381 

1 

6.671 

1 

0.001 

! 

40.001 

1 

2.331 

1 

7.411 

Induced  NMAC/  | 
NMACs  (%)1 

1 

100.001 

1 

83.331 

1 

100.001 

1 

0.001 

1 

100.001 

) 

100.001 

1 

100.001 

2  Reversals/  | 
Reversals  (%) 1 

i 

1.34  1 

1 

30.361 

1 

10.711 

1 

0.001 

1 

19.231 

1 

0.001 

1 

6.091 

NMACS/  1 

2  Rev.  {%) 1 

1 

0.00! 

1 

11.761 

1 

11.111 

1 

0.00! 

1 

40.001 

1 

0.00! 

1 

0,001 

Induced  NMAC/ I 
NMACs  (%)l 

! 

0.001 

1 

100.001 

1 

100.001 

1 

0.001 

1 

100.001 

1 

0.001 

1 

0.001 

3+  Reversals/ | 
Reversals  (%) 1 

1 

0.001 

1 

0.001 

1 

0.001 

1 

0.001 

1 

3.851 

1 

0.001 

1 

0.001 

NMACS/  1 

3+  Rev.  (%) 1 

1 

0.001 

1 

0.00! 

1 

0.001 

1 

0.001 

1 

100.001 

1 

0.001 

1 

0.001 

Induced  NMAC/ 1 
NMACs  { % ) 1 

1 

0.00! 

1 

0.001 

1 

0.00! 

1 

0.001 

1 

100.001 

1 

0.001 

1 

0.00! 

*  NMACs  and  average  alt.  sep.  at  CPA  are  based  on  simulation  truth 


Figure  3-1.  Multiple  TCAS-TCAS  Reversals. 
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Altitude  in  Feet 


ENCOUNTER  SUMMARY  -  AIRCRAFT  ALTITUDES 
Data  File  Name=II717yh.e05:  REtT  NumbeP-9046 

{Source:  LL  Composite  FTEG  Run,  Dated  :1 1/04/06 )  AO  Tune 


Figure  3-2.  Encounter  Summary  V7IR10  Multiple  Sense  Reversal. 


Table  3-1.  V7IR10-Mod  1  Reversed  Encounters  Planned  Separation 

vs.  Achieved  Separation 


Reversed  Enc, 

Rev.  NMAC 

Achieved  Sepan 

ition  (AC1  - 

AC2) 

Plan.  Sep. 

All 

Multi 

All 

Multi 

0 

.250 

0 

..(■250) 

0 

..500 

0 

..(-500) 

0 

..750 

0 

..(-750) 

0 

..1000 

0 

..(-1000) 

0 

344 

16 

7 

0 

10 

3 

37 

12 

250 

583 

90 

19 

6 

33 

25 

91 

73 

500 

846 

109 

258 

10 

280 

128 

326 

177 

750 

913 

69 

128 

8 

386 

66 

583 

132 

667 

161 

1000 

76 

0 

0 

0 

1 

0 

14 

0 

30 

0 

39 

0 

-250 

378 

45 

3 

0 

5 

14 

67 

48 

-500 

697 

77 

76 

3 

45 

131 

97 

198 

! 

-750 

940 

115 

41 

39 

56 

45 

97 

513 

137 

540 

-1000 

13 

0 

0 

0 

0 

0 

0 

3 

6 

4 

9 

4 

Total 

4790 

521 

532 

66 
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3.3.1  Resolution  of  Multiple  Reversal  Performance 

During  December  1996  and  January  1997  in  order  to  study  the  effect  of  reversals,  Lincoln 
Laboratory,  MITRE,  and  WJHTC  proposed  five  candidate  revisions  to  the  V7IR10-modl  logic. 
Mod  A  allowed  TCAS-TCAS  geometric  reversals  only  during  the  first  three  seconds  of  an 
encounter  when  the  low  ID  aircraft  wanted  to  switch  from  a  crossing  RA  to  a  non-crossing  RA. 
Mod  B  allowed  reversals  whenever  the  low  ID  aircraft  wanted  to  switch  from  a  crossing  RA  to  a 
non-crossing  RA.  Mod  C  contained  a  hard  limit  of  one  reversal.  Mod  D  did  not  allow  geometric 
TCAS-TCAS  reversals,  only  coordination  reversals  as  in  6.04a.  Mod  E  was  the  same  as  mod  B, 
but  with  a  hard  limit  of  one  reversal.  As  simulation  results  comparing  the  performance  of  these 
alternative  designs  became  available,  Lincoln  Laboratory,  MITRE,  WJHTC,  and  the  TCAS 
program  office  had  frequent  conference  calls  to  discuss  the  merits  and  drawbacks  of  all  five 
candidate  logic  modifications. 

Mod  C  was  eliminated  quickly  during  one  of  the  early  conference  calls  due  to  the  poor  results  of 
the  MITRE  safety  study.  The  six  encounter  classes  with  the  worst  performance  for  V7IR10- 
modl  (3/13,  6/16,  and  9/19)  were  chosen  as  test  cases.  WJHTC  collected  data  for  the  test  cases 
with  the  four  remaining  candidates  (A,  B,  D,  and  E).  Lincoln  Laboratory  and  WJHTC  studied  the 
results.  MITRE  performed  operational  acceptability  and  safety  studies  for  the  four  candidates  as 
well.  Due  to  extreme  time  pressure,  the  Lincoln  Laboratory  analysis  was  restricted  to  the  Hot- 
Spot  Program  and  Reversal  Analysis  Program  Variation  One.  These  two  programs  were  run 
using  all  12  test  data  files  (3  data  files  each  containing  two  encounter  classes,  four  logic 
versions).  The  tables  showing  the  outcome  of  reversed  encounters  by  planned  separation  and 
achieved  separation  were  produced  manually  for  candidates  A,  B,  and  E  only,  since  D  precluded 
TCAS-TCAS  reversals.  These  tables,  which  were  faxed  to  MITRE,  WJHTC  and  the  TCAS 
program  office  before  a  conference  call  showed  clearly  that  encounters  with  single  or  multiple 
reversals  still  had  an  overwhelming  tendency  to  lose  separation. 

Mod  A  was  eliminated  because  the  results  from  the  six  test  classes  were  poor.  Mod  B  was 
eliminated  because  the  safety  analysis  results  for  this  class  were  identical  to  the  safety  analysis 
results  for  Mod  E,  and  the  TCAS  program  office  considered  that,  without  some  added  benefit  in 
scenarios  where  both  pilots  were  already  responding,  multiple  TCAS-TCAS  reversals  were 
undesirable  from  a  human  factors  perspective.  This  left  Mod  D  and  Mod  E  as  "finalists". 
Neither  version  allowed  for  the  possibility  of  multiple  sense  reversals,  so  this  issue  was 
effectively  resolved. 

3.4  PERFORMANCE  WITH  SINGLE  REVERSALS 

By  studying  the  final  two  versions.  Mod  D  (no  reversals)  and  Mod  E  (reversals),  it  was  possible 
to  decide  if  TCAS-TCAS  geometric  reversals  should  be  included  in  the  Change  7  CAS  logic. 

Table  3-2  shows  planned  separation  vs.  achieved  separation  for  Mod  E  reversed  RAs.  Table  3-2 
shows  clearly  that  most  encounters  with  reversed  RAs,  even  those  that  did  not  result  in  NMACs, 
lost  vertical  separation.  For  example,  looking  at  Table  3-2,  for  Mod  E  there  are  230  reversed 
encounters  with  500  feet  of  planned  separation.  Note  that  99  of  these  encounters  resulted  in 
NMACs.  Of  these  230  encounters  205  (151  +  54)  achieved  less  than  500  feet  of  separation,  i.e., 
205  encounters  out  of  230  have  less  separation  with  TCAS  involved. 
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Table  3-2.  Mod  E  Reversed  Encounters  Planned  Separation  vs.  Achieved  Separation, 

Classes  3/13, 6/16,  and  9/19  Only 


Reversed  Enc. 

Rev.  NMAC 

Achieved  Separation  (AC1  - 

-AC2) 

Plan.  Sep. 

Al) 

Multi 

All 

Multi 

0 

0 

0 

0 

0 

0 

0 

0 

..250 

..(-250) 

..500 

..(-500) 

..750 

..(-750) 

..1000 

,.(-1000) 

0 

4 

0 

0 

0 

0 

0 

0 

3 

250 

2 

0 

1 

0 

2 

0 

0 

0 

500 

230 

0 

99 

0 

115 

54 

151 

54 

750 

200 

0 

0 

0 

16 

0 

142 

0 

196 

0 

24 

0 

1000 

25 

0 

0 

0 

0 

0 

9 

0 

22 

0 

-250 

38 

0 

8 

0 

4 

14 

0 

27 

-500 

285 

0 

112 

0 

49 

148 

51 

189 

0 

169 

-750 

177 

0 

1 

0 

1 

0 

114 

0 

10 

0 

10 

-1000 

10 

0 

0 

0 

0 

1 

0 

9 

Total 

971 

0 

221 

0 

Now  that  there  were  only  two  versions  of  the  Change  7  logic  under  study  (V7IR1 0-Mod  D  and 
V7IR1 0-Mod  E),  WJHTC  was  able  to  collect  data  for  sixteen  of  the  twenty  encounter  classes. 
Classes  0/10  and  1/11  were  not  run  because  they  have  no  NMACs.  A  limited  analysis  of 
candidates  D  and  E  was  performed  at  Lincoln  Laboratory.  Table  3-3  shows  the  percent  of 
encounters  with  induced  NMACs  tabulated  for  6.04a,  Mod  D-lOO,  Mod  E-lOO,  Mod  D-25,  and 
Mod  E-25.  Table  3-4  shows  the  percent  of  encounters  with  unresolved  NMACs  for  the  same 
five  versions.  Under  the  comments  column  in  Tables  3-3  and  3-4  the  weight  entry  in  square 
brackets  is  taken  from  Appendix  M,  Class  Weights,  converted  to  a  percentage  of  the  encounters 
observed  in  the  United  States  airspace. 

Note  that  each  class  weight  shown  in  Appendix  M  represents  the  percentage  of  pairwise 
encounters  out  of  the  total  ARTS  database  sampled  by  MITRE  in  the  1980s  that  exhibited  the 
geometry  of  that  class.  In  the  original  database,  within  each  class,  the  exact  breakdown  of 
aircraft  rates,  accelerations,  times  of  acceleration,  etc.  is  not  known.  For  the  WJHTC  simulation, 
for  each  class,  aircraft  rates  and  accelerations  were  used  which  spanned  a  range  slightly 
exceeding  the  range  of  expected  values.  In  addition,  in  the  WJHTC  simulation,  the  times  of 
acceleration  were  chosen  to  be  the  worst  possible  times,  i.e.,  20  or  25  seconds  prior  to  closest 
approach.  Thus,  it  is  not  known  how  well  the  class  weights  in  Appendix  M  apply  to  the  classes 
in  the  WJHTC  simulation.  Nevertheless,  the  weights  are  considered  a  useful  evaluation  tool  and 
are  considered  in  the  Lincoln  Laboratory  logic  evaluation. 


Looking  at  Table  3-3,  seven  of  the  sixteen  encounter  classes  have  the  same  number  of  induced 
NMACs  with  Mod  D  and  Mod  E.  Three  classes  have  mixed  behavior,  meaning  one  Mod  is 
better  with  the  100-foot  tracker  and  the  other  is  better  with  the  25-foot  tracker.  Three  classes  (7, 
13,  and  16)  with  a  combined  weight  of  13.39  percent  have  better  performance  with  Mod  D. 
Tlu-ee  classes  (15,  17,  and  19)  with  a  combined  weight  of  2.98  percent  have  better  performance 
with  Mod  E.  The  poor  performance  of  the  TCAS-TCAS  reversals  in  class  13  (the  "Dallas 
bump-up"  class),  the  fourth  most  common  encounter  class  in  the  United  States  airspace,  should 
be  noted  as  a  weakness  for  the  V7IR1 0-Mod  E  logic. 


Table  3-3.  Percent  of  Encounters  with  Induced  NMACs,  6.04a,  Mod  D  and  Mod  E  Both 

Pilots  Responding 


Class 

6.04a 

Mod  D-100 

Mod  E-100 

Mod  0-25 

Mod  E-25 

Comments 

[Weight] 

2 

1.71 

1.55 

1.55 

1.22 

1.22 

SAME 

3 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

4 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

5 

0.37 

0.30 

0.30 

0.33 

0.33 

SAME 

6 

0.15 

0.15 

0.15 

0.10 

0.10 

SAME 

7 

1.95 

1.93 

2.00 

1.04 

1.04 

□  BEST 
[0.02%] 

8 

1.65 

1.82 

1.69 

1.23 

1.25 

MIXED 

9 

0.28 

0.16 

0.19 

0.09 

0.08 

MIXED 

12 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

13 

0.10 

0.05 

0.38 

0.05 

0.30 

DBEST 

[10.51%] 

14 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

15 

0.73 

0.57 

0.49 

0.65 

0.49 

EBEST 

[2.05%] 

16 

0.04 

0.08 

0.16 

0.00 

0.00 

DBEST 

[2.56%] 

17 

0.25 

0.18 

0.16 

0.22 

0.11 

EBEST 

[0.19%] 

18 

0.41 

0.77 

0.96 

0.24 

0.17 

MIXED 

19 

0.49 

0.84 

0.38 

0.46 

0.26 

EBEST 

[0.74%] 
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Table  3-4.  Percent  of  Encounters  with  Unresolved  NMACs,  6.04a,  Mod  D, 

Mod  £  Both  Pilots  Responding 


Class 

6.04a 

Mod  D-100 

Mod  E-100 

Mod  D-25 

Mod  E-25 

Comments 

[Weight] 

2 

0.58 

0.00 

0.00 

0.00 

0.00 

SAME 

3 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

4 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

5 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

6 

0.81 

0.58 

0.58 

0.69 

0.64 

EBEST 

[0.29%] 

7 

0.16 

0.00 

0.00 

0.00 

0.00 

SAME 

8 

1.16 

0.23 

0.23 

0.23 

0.23 

SAME 

9 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

12 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

13 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

14 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

15 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

16 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

17 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

18 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

19 

0.00 

0.00 

0.00 

0.00 

0.00 

SAME 

Looking  at  Table  3-4,  fifteen  out  of  sixteen  encounter  classes  have  the  same  number  of 
unresolved  NMACs  for  Mod  D  and  Mod  E.  Class  6  has  better  performance  for  Mod  E,  but  this 
is  a  difference  of  hundredths  of  a  percent  meaning  there  is  no  significant  difference  between 
Mod  D  and  Mod  E  with  respect  to  imresolved  NMACs.  Therefore,  overall  performance  for  both 
pilots  responding  with  reversals  is  a  little  worse  than  without  reversals. 

As  mentioned  in  Section  3.1,  the  main  reason  for  allowing  TCAS-TCAS  reversals  was  to 
provide  some  protection  against  a  non-responding  intruder.  If  the  reversals  in  Mod  E  achieved 
this  goal,  then  perhaps  this  would  be  reason  enough  to  allow  reversals,  despite  the  slightly  worse 
performance  against  responding  intruders. 


As  mentioned  earlier  in  Section  1.1.2,  WJHTC  collected  a  second  set  of  data  containing 
encounters  where  one  pilot  does  not  respond  to  the  TCAS  RA.  These  data  only  contain  6.04a 
and  C7-25  data.  The  pilot  non-responding  data  were  processed  for  Mod  D  and  Mod  E  using  the 
Hot-Spot  Program.  The  results  are  summarized  below  in  Table  3-5  (Induced  NMACs)  and 
Table  3-6  (Unresolved  NMACs). 
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Table  3-5.  Percent  of  Induced  NMACs,  6.04a,  Mod  D-25,  and  Mod  E-25, 

One  Pilot  Non-Responding 


Class 

6.04a 

One  NR 

C7-25D 

One  NR 

C7-25E 

One  NR 

6.04a/ 

C7-25DNR 

6.04a/ 

C725ENR 

C7-25D/ 

6.04aNR 

C7-25E/ 

6.04aNR 

2 

11.662 

7.764 

7.539 

11.211 

11.082 

7.861 

7.668 

3 

0.135 

0.709 

0.709 

0.135 

0.135 

0.676 

0.676 

4 

1.974 

0.987 

0.987 

1.974 

2.303 

0.987 

0.987 

5 

5.351 

3.774 

3.787 

5.218 

5.281 

3.774 

3.774 

6 

4.141 

2.646 

2.595 

4.448 

4.410 

2.518 

2.480 

7 

14.399 

9.277 

9.093 

14.045 

13.996 

9.028 

8.864 

8 

7.655 

6.322 

6.112 

7.817 

7.765 

6.30 

6.138 

9 

1.379 

1.449 

1.433 

1.160 

1.121 

1.542 

1.503 

12 

0.192 

0.048 

0.048 

0.192 

0.192 

0.048 

0.048 

13 

2.024 

2.050 

1.556 

1.923 

1.594 

2.176 

1.733 

14 

0.00 

0.00 

0.00 

0.00 

0.00 

0.781 

0.391 

15 

0.546 

0.445 

0.634 

0.485 

0.303 

0.485 

0.445 

16 

2.909 

2.752 

0.845 

2.653 

2.044 

2.732 

1.553 

17 

0.318 

0.563 

0.400 

0.527 

0.436 

0.490 

0.436 

18 

6.304 

4.581 

2.507 

6.545 

4.966 

4.532 

3.218 

19 

5.126 

3.917 

1.803 

4.194 

2.830 

4.160 

2.911 

Comparing  the  performance  of  Mod  D  and  Mod  E  for  the  C7-25  one  non-responding  (NR) 
columns,  for  the  planned  crossing  classes  Mod  E  has  better  performance  in  five  cases.  However, 
these  are  only  tenths  or  hundredths  of  a  percent  improvements;  Mod  D  had  better  performance  in 
only  one  case.  Comparing  Mod  D  and  Mod  E  for  the  C7-25  one  NR  columns,  for  the  planned 
non-crossing  classes  Mod  D  was  better  than  Mod  E  in  only  one  case  (class  15).  However, 
Mod  E  was  better  in  five  classes,  with  18  and  19  showing  the  most  dramatic  difference. 

Comparing  the  performance  of  Mod  D  and  Mod  E  in  Table  3-5  for  the  6.04a  /  C7-25  NR 
colmnns.  Mod  D  is  better  in  only  one  class  (14),  Mod  E  is  better  in  1 1  classes,  with  18  and  19 
again  showing  the  most  dramatic  improvement.  Comparing  the  C7-25  /  6.04aNR  columns, 
whenever  there  is  a  difference  in  performance  Mod  E  is  better,  and  the  plarmed  non-crossing 
classes  show  the  most  improvement. 
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Table  3-6.  Percent  of  Unresolved  NMACs,  6.04a,  Mod  D-25,  and  Mod  E-25, 

One  Pilot  Non-Responding 


Class 

6.04a 

One  NR 

C7-2SD 

One  NR 

C7-25E 

One  NR 

6.04a/ 

C7-25DNR 

6.04a/ 

C7-25ENR 

C7-25D/ 

6.04aNR 

C7-25E/ 

6.04aNR 

2 

2.315 

1.968 

1.968 

1.678 

1.678 

2.431 

2.431 

3 

0.543 

0.00 

0.00 

0.543 

0.543 

0.00 

0.00 

4 

6.667 

6.667 

6.667 

6.667 

6.667 

6.667 

6.667 

5 

7.946 

8.543 

8.386 

8.009 

8.197 

8.354 

8.040 

6 

3.877 

3.762 

3.848 

3.443 

3.675 

3.646 

3.559 

7 

8.416 

8.663 

8.622 

6.972 

7.013 

9.158 

8.787 

8 

4.090 

2.701 

2.758 

2.894 

3.086 

3.356 

3.337 

9 

1.986 

1.036 

1.036 

1.554 

1.554 

1.123 

1.123 

12 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

13 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

14 

33.333 

33.333 

33.333 

33.333 

33.333 

33.333 

33.333 

15 

30.882 

30.882 

30.882 

31.618 

31.618 

30.147 

30.147 

16 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

17 

22.619 

23.214 

23.214 

23.214 

23.214 

22.619 

22.619 

18 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

19 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Looking  at  Table  3-6,  all  of  the  planned  non-crossing  unresolved  NMAC  columns  are  identical 
for  Mod  D  and  Mod  E.  Class  15  shows  a  degradation  in  performance  in  the  6.04a  /  C7-25  NR 
columns.  Class  17  shows  a  degradation  in  performance,  relative  to  6.04a,  in  the  C7-25  one  NR 
and  6.04a  /  C7-25NR  columns.  Four  of  the  planned  crossing  classes  (2,  3,  4,  and  9)  show  no 
difference  in  performance  between  Mod  D  and  Mod  E,  while  the  other  classes  (5,  6,  7,  and  8) 
have  mixed  performance. 

Lincoln  Laboratory  pointed  out  to  the  TCAS  community  the  relatively  poor  performance  of 
TCAS-TCAS  reversals  in  class  13  (the  "Dallas  bump  up"  class)  and  the  marginal  improvement 
in  only  five  of  the  twenty  encounter  classes  for  a  non-responding  intruder  in  TCAS-TCAS 
coordinated  encounters.  Lincoln  Laboratory  also  pointed  out  that  the  overall  performance  for 
both  pilots  responding  was  worse  with  reversals  than  without  reversals.  Based  on  the  MITRE 
safety  study  results,  and  the  lack  of  overwhelming  evidence  for  the  removal  of  TCAS-TCAS 
reversals,  the  TCAS  community  decided  to  proceed  with  Mod  E.  (The  subject  of  reversal 
performance  will  be  revisited  in  Section  3.6.)  Once  the  decision  was  made  to  keep 
TCAS-TCAS  reversals,  Lincoln  Laboratory  was  tasked  to  perform  the  full  logic  evaluation  on 
the  complete  set  of  encounter  classes  for  logic  candidate  E,  which  became  known  as  Version  7 
Mod  10.  This  effort  was  delayed  by  work  necessary  to  prepare  the  DO-185A  Minimum 
Operational  Performance  Standards  (MOPS)  for  approval  by  SC-147. 
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3.5  FULL  LOGIC  EVALUATION 

In  April  1997,  after  the  completion  of  the  MOPS  work,  the  WJHTC  data  for  all  20  encounter 
classes  for  Version  7  Mod  10  were  analyzed  using  Lincoln  Laboratory’s  high  level  analysis 
tools:  the  Reversal  Analysis  Programs,  the  Hot-Spot  Program  and  the  Performance  Statistics 
Program.  Once  the  high  level  results  were  studied,  we  were  able  to  proceed  to  the  in-depth 
NMAC  analysis.  For  every  Change  7  NMAC  encounter,  a  set  of  three  plots  was  produced:  fhe 
encounter  with  6.04a/6.04a,  the  encounter  with  C7-100/C7-100,  and  the  encounter  with 
C7-25/C7-25.  The  encounter  summary  pages  were  also  produced  for  each  encounter  plotted  to 
allow  comparison  of  the  behavior  of  all  three  versions  of  the  logic. 

In  order  to  understand  the  failures  better,  these  encounter  plots  and  associated  summaries  were 
sorted  into  several  groups.  The  first  group  contained  encounters  where  all  three  logic  versions 
had  an  NMAC.  This  group  was  not  alarming  since  the  behavior  of  Version  7  Mod  10  was  no 
worse  than  6.04a. 

A  second  group  contained  encounters  where  C7-100  did  not  have  an  NMAC,  but  C7-25  did  have 
an  NMAC.  In  this  group  the  6.04a  encounters  were  mixed;  some  had  NMACs  some  did  not. 
However,  the  important  factor  is  that  the  25-foot  tracker,  which  one  would  expect  to  have  better 
performance  than  the  100-foot  tracker,  did  not  always  live  up  to  expectations. 

A  third  group  was  identified  where  both  Change  7  versions  had  NMACs,  but  6.04a  did  not  have 
an  NMAC.  This  group  was  studied  in  depth  because  the  Change  7  performance  was  worse  than 
the  6.04a  performance  and  because  several  of  the  failures  came  from  Class  13,  a  planned  non¬ 
crossing,  statistically  significant  group  of  encounters. 

From  these  studies,  Lincoln  Laboratory  identified  some  specific  areas  of  performance  weakness 
with  the  Version  7  Mod  10  TCAS-TCAS  logic.  These  areas  were  discussed  with  the  TCAS 
program  office,  MITRE,  WJHTC,  and  our  European  colleagues  after  SC-147  in  May  of  1997. 
The  Centre  D'Etudes  de  la  Navigation  Aerienne/  Direction  Generale  de  L' Aviation  Civile. 
(CENA/DGAC)  ACAS  representatives  to  SC- 147  had  previously  identified  weaknesses  with  the 
TCAS-unequipped  performance  based  on  their  analysis  of  Version  7  Mod  10.  It  was  agreed  that 
both  areas  of  weakness  should  be  examined  to  determine  if  further  revision  of  the  Change  7  logic 
was  required. 

3.6  RESOLUTION  OF  SINGLE  REVERSAL  PERFORMANCE 

During  the  early  stages  of  the  TCAS-TCAS  reversal  analysis  work,  (see  Section  2.1)  one  of  the 
most  powerful  analytical  tools  available  was  the  table  showing  the  achieved  separation  for 
reversed  encounters  grouped  by  planned  separation.  The  computation  of  these  tables  was  time 
consuming,  so  the  process  was  automated  in  the  Separation  Differences  program,  also  known  as 
the  Reversal  Analysis  Program  Variation  Two.  For  this  program,  the  NMAC  was  replaced  by 
another  measure  of  performance:  the  separation  difference.  The  separation  difference  is  defined 
as  the  absolute  value  of  the  achieved  separation  minus  the  absolute  value  of  the  planned 
separation.  Here,  a  positive  value  indicates  that  we  gained  separation,  a  negative  value  means 
we  lost  separation.  For  example,  if  two  aircraft  achieved  -250  feet  of  separation  with  TCAS  and 
they  had  a  planned  separation  of +500  feet,  the  separation  difference  is  abs(-250)  -  abs(500)  or 
-250  feet.  Here,  the  aircraft  were  250  feet  closer  with  TCAS  than  they  would  have  been  without 
TCAS  -  a  poor  result. 
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In  order  to  be  able  to  fairly  compare  separation  losses  and  gains  for  reversed  encounters  and  non- 
reversed  encounters  two  sets  of  four  tables  were  produced  for  every  encounter  class.  The  first 
set  of  tables  is  for  encounters  with  non-reversed  RAs,  the  second  set  of  tables  is  for  encounters 
with  reversed  RAs.  Each  set  contains  four  tables,  showing  the  number  of  (1)  planned  crossing 
encounters  with  separation  gains,  (2)  planned  crossing  encounters  with  separation  losses,  (3) 
planned  non-crossing  encounters  with  separation  gains,  and  (4)  planned  non-crossing  encounters 
with  separation  losses.  Each  table  shows  the  data  tabulated  by  planned  separation  and  achieved 
separation.  This  combination  of  tables  enabled  Lincoln  Laboratory  to  pinpoint  planned 
separation  values  where  the  logic  is  more  trouble-prone. 

These  separation  difference  tables  allowed  us  to  answer  our  earlier  question,  "Are  the 
TCAS-TCAS  reversals  helping"?  During  the  full  logic  evaluation  of  Version  7  Mod  10  the 
answer  was,  "No,  the  TCAS-TCAS  reversals  are  not  helping".  Few  reversed  encounters 
increased  separation  (eleven  percent)  and  most  reversals  in  the  WJHTC  data  caused  separation 
loss.  As  an  example.  Figure  3-3  shows  the  results  for  Class  13,  one  of  the  more  common  non¬ 
crossing  classes  in  the  U.S.  airspace.  In  this  example,  for  non-reversed  RAs  only  30  encountCTS 
out  of  9944  (0.3%)  lost  separation,  there  was  only  1  NMAC,  and  many  encounters  significantly 
increased  separation.  In  contrast  for  reversed  RAs  100%  lost  separation,  there  were  68  NMACs, 
and  there  were  no  encounters  with  separation  gains.  This  discouraging  result  was  shared  during 
a  conference  call  with  the  TCAS  program  office,  MITRE  and  WJHTC. 

During  the  summer  of  1997,  both  WJHTC  and  MITRE  proposed  modifications  to  the  Version  7 
Mod  10  TCAS  logic  to  address  the  reversal  performance  problem.  The  WJHTC  proposal  was 
targeted  for  TCAS-TCAS  encounters,  it  allowed  own  aircraft  to  reverse  sense  earlier  if  it 
determined  that  own  aircraft  had  maneuvered  contrary  to  the  sense  of  the  RA  during  the  pilot 
response  delay.  Both  WJHTC  and  Lincoln  Laboratory  were  encouraged  by  the  results  of  this 
modification  The  Separation  Differences  program  (Reversal  Analysis  Program  Variation  Two) 
was  run  on  the  test  data  and  the  improvement  was  dramatic.  The  reversed  encounters  showed 
significant  gains  in  separation,  and  the  separation  losses  for  reversed  encounters  were 
comparable  to  those  in  non-reversed  encounters.  This  improvement  did  not  come  without  a  cost 
-  an  increase  in  the  number  of  reversed  encounters,  but  this  was  considered  acceptable  because 
the  reversed  encounters  no  longer  tended  to  lose  vertical  separation  as  before. 

The  proposed  MITRE  modification  was  targeted  for  the  TCAS-unequipped  encounters.  This 
modification  was  tested  mostly  by  MITRE  and  our  CENA/DGAC  ACAS  colleagues  and  result^ 
in  a  one  geometric  reversal  cap  on  TCAS-unequipped  encounters.  In  August  1997,  Lincoln 
Laboratory,  MITRE,  WJHTC,  the  TCAS  program  office,  plus  CENA/DGAC  and  Defense 
Evaluation  and  Research  Agency  (DERA)  ACAS  representatives  agreed  to  accept  both  the 
WJHTC  and  MITRE  modifications.  The  designation  for  this  baseline  of  the  logic  was  Version  7 
Mod  11. 
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NON  REVERSED  RAs 
REVERSED  RAs 

GAINS  PLAN  SEP  |  NO  GAIN 
NON-REVERSED  0  FT|  0 

250  FTl  0 

500  FTl  894 

750  FTl  144 

1000  FTl  140 


GAINS  PLAN  SEP  1  NO  GAIN 


REVERSED  0  FT | 

250  FTl 
500  FTi 
750  FTl 
1000  FTl 


LOSSES  PLAN  SEP  j 
NON-REVERSED  0  FT | 
250  FTl 
500  FTl 
750  FTl 
1000  FTl 


LOSSES  PLAN  SEP  I 
REVERSED  0  FT| 
250  FTl 
500  FTl 
750  FTl 
1000  FTl 


RA  REVERSAL  RESULTS 
CLASS  13 

CHANGE  7  MOD  10 

RA  COUNT  9944 
RA  COUNT  87 


>0. .250 
0 

114 

1116 

626 

243 


>0. .250 
0 
0 
0 
0 
0 


>0. .250 
0 

16 

13 

0 

0 


>0. .250 
0 
0 
5 
0 
0 


>250. .500 
54 

2018 

886 

513 

299 


>250. .500 
0 
0 
0 
0 
0 


>250. .500 
0 
0 
1 
0 
0 


>250. .500 
0 
0 

82 

0 

0 


NMAC  COUNT  1 
NMAC  COUNT  68 


>500. .750 
422 
918 
494 
228 
84 


>500. .750 
0 
0 
0 
0 
0 


>500. .750 
0 
0 
0 
0 
0 


>500. .750 
0 
0 
0 
0 
0 


>750. .1000 
36 
312 
181 
88 
4 


>750. .1000 
0 
0 
0 
0 
0 


>750. 


.1000 

0 

0 

0 

0 

0 


>750. .1000 
0 
0 
0 
0 
0 


>  1000 
0 
46 
54 
0 
0 


>  1000 
0 
0 
0 
0 
0 


>  1000 
0 
0 
0 
0 
0 


>  1000 
0 
0 
0 
0 
0 


Figure  3-3.  Reversal  Analysis  Program  Variation  Two  -  Separation  Differences 

Version  7  Mod  10. 


Lincoln  Laboratory  performed  a  complete  evaluation  of  the  Version  7  Mod  11  logic  for  all 
twenty  classes.  The  most  dramatic  improvement  is  shown  in  Figure  3-4,  the  Separation 
Differences  results  for  class  B.  The  reversed  encounters  now  have  less  separation  losses  than  for 
Mod  10,  and  those  losses  are  smaller.  The  reversed  encounters  now  have  many  significant 
separation  gains  that  were  not  present  for  Mod  10.  This  improvement  in  performance  was  seen 
also  in  the  hot-spot  tables.  For  Version  7  Mod  10  several  classes  (6,  13,  16,  and  18)  had  higher 
induced  NMAC  counts  than  6.04a,  but  with  Version  7  Mod  11  for  all  classes,  with  no 
exceptions,  we  see  a  reduction  in  induced  NMACs  compared  to  6.04a. 
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RA  REVERSAL  RESULTS 
CLASS  13 


CHANGE  7  MOD  11  (MITRE  +  WJHTC  mods) 

NON  REVERSED  RAs  RA  COUNT  914  9  NMAC  COUNT  0 

REVERSED  RAs  RA  COUNT  856  NMAC  COUNT  6 


GAINS  PLAN  SEP  1  NO  GAIN  |  >0..250 


NON-REVERSED  0 

FTl 

0  1 

0 

250 

ETi 

0  ! 

37 

500 

FTl 
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750 

FT| 

146  1 
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Figure  3-4.  Reversal  Analysis  Program  Variation  Two  -  Separation  Differences 

Version  7  Mod  11. 


4.  GENERAL  ANALYSIS 


4.1  DESCRIPTION 

The  second  goal  of  the  Change  7  (Version  7  Mod  11)  logic  analysis  using  the  Lincoln 
Laboratory  analysis  programs  was  to  detect  and  explain  any  areas  of  poor  performance 
remaining  in  the  Change  7  TCAS  logic.  These  "areas  of  poor  performance"  were  defined  in 
Section  2.3  to  be  any  matrix  cell  for  which  logic  Version  7  Mod  1 1  had  more  NMACs  than 
6.04a,  the  current  baseline. 

4.2  RESULTS  -  BOTH  PILOTS  RESPONDING 

4.2.1  Hot-Spot  and  Matrix  Generator  Programs 

There  were  no  hot-spots  for  TCAS-TCAS  encounters  where  both  aircraft  were  Version  7 
Mod  11  equipped. 

The  hot-spot  tables  are  useful  for  locating  interoperability  problems.  In  general  the  intermix 
performance  fell  between  that  of  the  corresponding  single  versions  (6.04a/6.04a,  C7-100/C7-100, 
or  C7-25/C7-25).  There  are  three  exceptions  to  this  pattern.  In  Table  5.3,  there  are  unresolved 
NMACs  for  6.04a/C7-100  (0.063%)  where  there  are  no  NMACs  for  either  6.04a/6.04a  or  C7- 
100/C7-100  This  falls  below  the  2%  significance  threshold,  so  it  is  not  a  true  hot-spot.  In 
Table  7-3,  the  percent  of  unresolved  NMACs  for  6.04a/C7-100  (0.206%)  is  higher  than 
6.04a/6.04a  (0.165%)  and  C7-100/C7-100  (0.0%).  This  represents  a  25%  increase  in  NMACs, 
but  still  falls  below  the  2%  significance  threshold,  so  it  is  not  a  true  hot-spot.  Finally,  in  Table 
17.4  the  percent  of  induced  NMACs  is  higher  for  C7-25/C7-100  (0.136%)  than  for  C7-25/C7-25 
(0.109%)  and  C7-100/C7-100  (0.127%).  This  represents  an  8%  increase  in  NMACs,  but  still 
falls  below  the  2%  significance  threshold,  so  it  is  not  a  true  hot-spot  either. 

Figures  4-1  and  4-2  show  histograms  of  the  number  of  NMACs  (as  a  percentage  of  the  number 
of  planned  encounters)  for  each  of  the  20  encounter  classes  for  all  three  versions  of  the  TCAS 
logic.  Figure  4-1  shows  induced  NMACs  and  Figure  4-2  shows  unresolved  NMACs.  As  you 
can  see  in  both  Figures  4-1  and  4-2  Version  7  Mod  11  consistently  performs  as  well  as,  or  better 
than  6.04a. 

4.2.2  NMAC  Characterization  Programs 

Since  there  were  no  TCAS-TCAS  hot-spots  left  for  Version  7  Mod  11,  the  analysis  programs 
that  were  previously  used  to  understand  hot-spots  were  run  over  all  of  the  data.  The  output  from 
these  programs  was  useful  in  studying  the  Version  7  Mod  1 1  Representative  NMACs,  which  will 
be  described  in  Section  6  of  this  report. 
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Figure  4-2.  Unresolved  NMACs. 

4.2.2.1  NMACs  as  a  Function  of  Parameter  Value 

Appendix  H  contains  the  results  of  NMAC  Characterization  Program  One  (NMACs  as  a  function 
of  parameter  values)  for  all  twenty  encounter  classes  for  Version  7  Mod  11.  A  sample  output 
was  provided  in  Figure  2-7.  The  first  two  values  in  each  line  identify  the  encounter  class  and 
matrix  table.  Note  that  in  the  aircraft  1  equipage  column  (ACl  EQ)  and  the  aircraft  2  equipage 
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column  (AC2  EQ),  65  represents  version  6.04a,  71  represents  C7-100,  and  75  represents  C7-25. 
The  parameters  found  in  NMACs  for  a  given  matrix  table  and  equipage  are  printed,  along  with  a 
count  of  the  number  of  times  the  parameter  value  occurred. 

From  the  printout  in  Appendix  H,  we  can  answer  questions  about  the  NMACs  of  6.04a  or 
Version  7  Mod  1 1  fairly  easily.  For  example,  "What  was  the  distribution  of  planned  sepa.rations 
for  Version  7  Mod  11  NMACs"?  Scanning  through  Appendix  H,  Table  4-1  summariang  the 
absolute  value  of  the  planned  separation  by  table  number  for  all  Version  7  Mod  11  NMACs  is 
easily  constructed.  From  Appendix  A,  we  observe  that  only  classes  3/13  and  9/19  have  planned 
encounters  with  +/-  1000  feet  of  separation.  Entries  in  Table  4-1  for  classes  without  planned 
encounters  of +/- 1000  feet  of  separation  are  indicated  by  a  dash  (-). 


From  Table  4-1,  we  can  state  that  for  Version  7  Mod  11  there  were  no  NMACs  with  a  planned 
separation  of +/-  1000  feet.  This  is  important  because  1000  feet  is  the  normal  separation  in  the 
airspace  and  previous  versions  of  the  TCAS  logic  did  have  problems  with  these  encounters. 


Table  4-1.  Distribution  of  NMACs  by  Absolute  Value  of  Planned  Separation 


Similarly,  we  may  ask,  "What  is  the  distribution  of  intruder  aircraft  vertical  rates  for  Change  7 
NMACs"?  From  Appendix  H,  we  can  generate  Table  4.2,  which  shows  the  vertical  rate  of  the 
intruder  aircraft  for  C7-100/C7-100  and  C7-25/C7-25  NMAC  encounters.  From  Table  4.2  we 
can  conclude  that  nearly  all  Version  7  Mod  11  NMACs  the  intruder  had  vertical  rate  of 
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+/-  3000  fpm  or  +/-  5000  fpm.  The  only  exceptions  to  this  are  four  NMACs  in  Table  7.4  and 
three  NMACs  in  Table  18.4,  all  with  vertical  rates  of -1000  fpm.  This  is  consistent  with  earlier 
versions  of  the  logic. 


Table  4-2.  Distribution  of  Intruder  Aircraft  Vertical  Rates 


Version  7  Mod  1 1  100-foot  Rate  (fpm) 

Version  7  Mod  1 1  25-foot  F 

late  (fpm) 

Table 

+/- 1000 

+/-  3000 

+/-  5000 

+/-1000 

+/-  3000 

+/-  5000 

2.4 

0 

0 

14 

0 

0 

18 

5.4 

0 

0 

4 

0 

1 

6 

6.3 

0 

0 

6 

0 

1 

5 

6.4 

0 

0 

12 

0 

0 

4 

7.4 

4 

14 

66 

0 

3 

32 

8.3 

0 

0 

6 

0 

0 

4 

8.4 

0 

2 

87 

0 

0 

65 

9.4 

0 

0 

5 

0 

0 

2 

13.4 

0 

0 

3 

0 

0 

0 

15.4 

0 

0 

2 

0 

0 

2 

17.4 

0 

6 

1 

0 

6 

0 

18.4 

3 

4 

2 

0 

1 

0 

19.4 

0 

2 

11 

0 

1 

2 

4.2.2.2  NMACs  as  a  Function  of  Logic  Version 

Three  different  comparisons  of  NMACs  by  logic  version  were  generated.  Figure  4-3  provides 
the  results  for  the  6.04a/6.04a  and  C7-100/C7-100  comparison.  Looking  at  Figure  4-3,  we  can 
make  several  observations.  C7-100  eliminated  all  NMACs  that  were  present  in  Tables  2.3,  7.3, 
and  16.4  for  6.04a.  None  of  the  NMACs  in  Tables  13  .4  and  18.4  are  common  to  both  6.04a  and 
C7-100.  There  were  no  tables  where  C7-100  had  NMACs  where  6.04a  had  none. 
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Figure  4-3.  NMAC  Comparison  6.04a  vs.  Change  7  100-foot  tracker. 

Figure  4-4  provides  the  results  of  the  6.04a/6.04a  and  C7-25/C7-25  comparison.  Looking  at 
Figure  4-4  we  can  make  several  observations.  C7-25  eliminated  all  NMACs  that  were  present  for 
6.04a  in  Tables  2.3,  7.3,  13.4,  and  16.4.  None  of  the  NMACs  in  Tables  6.4  and  19.4  are 
common  to  6.04a  and  C7-25.  There  were  no  tables  where  C7-25  had  NMACs  where  6.04a  had 

none. 


Figure  4-4.  NMAC  Comparison  6.04a  vs.  Change  7  25-foot  tracker. 

Figure  4-5  provides  the  results  of  the  C7-100/C7-100  and  C7-25/C7-25  comparison.  Looking  at 
Figure  4-5  we  can  make  several  observations.  One  observation  is  that  for  Tables  6.4,  18.4,  and 
19.4  there  are  no  NMACs  in  common.  In  most  cases,  the  25-foot  tracker  has  fewer  NMACs  than 
the  100-foot  tracker  as  expected;  the  exceptions  to  this  are  Table  6.3  and  15.4  where  the  NMAC 
totals  are  the  same  for  both  trackers  and  Tables  2.4  and  5.4  where  the  25-foot  tracker  logic  has 
more  NMACs  than  the  100-foot  tracker  logic. 
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It  seems  vinusual  that  the  encounters  with  the  100-foot  tracker  would  have  a  better  outcome  than 
encounters  with  the  25-foot  tracker.  In  order  to  understand  how  this  could  occur,  the  encounter 
summaries  from  Tables  2.4  and  5.4  where  Change  7-25  had  an  NMAC  and  Change  7-100  did  not 
have  an  NMAC  were  examined. 

In  the  Tables  10  and  11,  there  are  a  total  of  twelve  encounters  in  which  Change  7-25  had  an 
NMAC,  but  Change  7-100  did  not  have  an  NMAC  (eight  encounters  in  Table  2.4  and  four 
encounters  in  Table  5.4).  In  all  twelve  encounters,  both  logic  versions  picked  the  same  initial 
RA.  After  that,  there  was  wide  variation  in  the  presence  and/or  timing  of  a  follow-on  RA 
(increase  or  reversal)  with  no  correlation  to  logic  version.  Lincoln  Laboratory  personnel 
discussed  this  behavior  with  the  developer  of  MITRE’s  25-foot  and  100-foot  vertical  packers.  It 
was  MITRE’s  sense  that  the  variation  in  the  follow-on  RAs  occurred  because  of  the  interaction 
between  the  altitude  quantization  and  the  thresholds  in  the  detection  logic.  Therefore,  this  is  not 
a  deficiency  in  the  25-foot  tracker. 

For  Table  5.4,  there  are  four  encounters  where  Change  7-100  did  not  have  an  NMAC,  but 
Change  7-25  did  have  an  NMAC.  In  all  four  cases  the  difference  was  that  Change  7-25  did  not 
strengthen  the  initial  RA. 


Figure  4-5.  NMAC  Comparison  Change  7  100-foot  tracker  vs.  Change  7  25-foot  tracker. 
4.23  Performance  Statistics  Program  Results 

The  Performance  Statistics  Program,  described  in  Section  2,  provides  information  on  the 
frequency  and  effectiveness  of  altitude  crossing  TCAS  advisories.  Table  4-3  summarizes  the 
two  most  significant  statistics  generated  for  all  twenty  encounter  classes;  the  percent  of  RAs  that 
were  altitude  crossing  and  the  percent  of  altitude  crossing  RAs  that  were  NMACs.  Remember, 
for  classes  0  -  9  the  aircraft  intended  to  cross  in  altitude,  for  classes  10  -  19  the  aircraft  did  not 
plan  to  cross  in  altitude. 

The  number  of  crossing  RAs  for  Version  7  Mod  11  is  comparable  to  or  greatly  reduced 
compared  to  6.04a  with  two  exceptions.  First,  encounter  class  15,  C7-100  has  a  slight  increase  in 
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altitude  crossing  RAs.  Also,  for  encounter  class  3,  both  C7-100  and  C7-25  have  increased 
crossing  RAs  (41%  and  54%,  respectively).  Class  3  is  a  planned  crossing  class,  so  an  increase  in 
altitude  crossing  RAs  is  not  a  cause  for  concern.  The  increase  in  altitude  crossing  RAs  for 
classes  3  and  15  is  not  alarming  because  the  percent  of  crossing  RAs  resulting  in  NMACs  for 
encounter  class  3  remains  at  0  for  both  C7-100  and  C7-25  and  the  percent  of  crossing  RAs 
resulting  in  NMACs  for  encounter  class  15  is  greatly  reduced  for  both  C7-100  (82%  less  than 
6.04a)  and  C7-25  (78%  less  than  6.04a). 

Looking  at  the  percent  of  crossing  RA  NMACs,  there  are  several  encounter  classes  (2,  5,  8,  and 
16)  that  had  crossing  RA  NMACs  with  6.04a,  but  none  for  either  C7-100  or  C7-25.  From 
Table  4-3  we  can  conclude  that  Version  7  Mod  1 1  reduced  the  overall  number  of  crossing  RAs, 
and  those  crossing  RAs  that  remain  are  more  effective,  i.e.,  they  result  in  fewer  NMACs. 


Table  4-3.  Summary  of  Performance  Statistics  Results 


4.2.4  Reversal  Analysis  Programs 


4.2.4.1  Variation  One 

After  the  multiple  TCAS-TCAS  reversals  were  eliminated,  and  the  MITRE  and  WJHTC 
improvements  were  made  to  the  reversal  logic,  Lincoln  Laboratory  ran  the  Reversal  Analysis 
Program  Variation  One  for  all  twenty  encounter  classes.  The  complete  set  of  results  for  the 


44 


Reversal  Analysis  Program  Variation  One  are  in  Appendix  C.  Since  TCAS-TCAS  reversals 
were  not  allowed  in  6.04a,  no  meaningful  comparison  of  reversal  performance  can  be  made  from 
that  perspective. 

Reversals  against  unequipped  threats  were  allowed  in  6.04a,  so  we  can  compare  the  performance 
of  Version  7  Mod  11  to  6.04a  for  the  TCAS-unequipped  cases.  Table  4-4  summarizes  both  the 
percent  of  RAs  that  reversed  and  the  percent  of  reversed  RA  NMACs  for  6.04a  vs.  Mode  C,  C7- 
100  vs.  Mode  C,  and  C7-25  vs.  Mode  C.  Under  the  comments  column  "Same",  "Better",  and 
"Worse"  refer  to  the  performance  of  Change  7  compared  to  6.04a.  Also  under  comments,  the 
[Class  Weight]  entry  is  taken  from  Appendix  M,  converted  to  a  percentage  of  the  encounters 
observed  in  the  United  States  airspace.  For  example,  in  the  row  for  class  9,  Worse  [0.01]  means 
that  the  Change  7  performance  was  worse  than  6.04a  (because  there  were  more  reversed  RAs 
that  were  NMACs)  and  class  9  represents  0.01  percent  of  the  encounters  in  the  airspace. 

Table  4-4.  Effectiveness  of  Reversals  Against  Unequipped  Threats 


F 

’ercent  of  FRAs 
'hat  Reversed 

Percent  Reversed  RAs 

That  Were  NMACs 

Comments 

Class 

6.04a 

100  ft 

25  ft 
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25  ft 
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0 

0.00 

0.00 

0.00 
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1 

0.00 

0.00 

0.00 

0.00 

0.00 

wamm 

Same 

2 

2.06 

5.18 

5.88 

0.00 

0.98 

2.16 
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[0.04%] 

3 

1.60 

0.82 

0.00 

0.00 

0.00 

4 

0.00 

0.00 

OOO 

Same 

5 

2.70 

3.57 

2.04 
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[0.25%] 

6 

1.40 

2.34 

2.50 

3.17 

2.37 

2.21 
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7 

2.67 

7.59 

9.09 

2.73 

2.44 

2.66 
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[0.02%] 

8 
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7.51 

7.49 

3.69 

2.43 

2.22 
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9 

2.32 

3.85 

2.70 
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2.50 

2.88 
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[0.01%] 

10 

0.00 

0.00 
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0.00 

0.00 

0.00 
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11 

0.00 

0.00 

0.00 

0.00 

0.00 
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12 

1.25 

0.30 

0.00 

0.00 

13 

11.62 

13.30 

16.09 

0.00 

0.00 

14 
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16 
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18 
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imiiiiP^^iiiii 
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19 

18.58 

15.35 

miiiimii 
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[0.74%] 
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Looking  at  the  percent  of  RAs  that  reversed,  some  classes  still  had  no  reversed  RAs  for 
Version  7  Mod  1 1  (0, 1, 4,  10, 1 1,  and  14).  Some  classes  had  fewer  reversed  RAs  than  6.04a  (3, 
12,  16,  and  19).  Class  18  had  more  reversed  RAs  with  Change  7-100  and  fewer  reversed  RAs 
with  Change  7-25.  The  rest  of  the  encounter  classes  (2,  5,  6,  7,  8,  9,  13,  15,  and  17)  had  more 
reversed  RAs  with  Version  7  Mod  1 1  than  with  6.04a.  This  result  in  itself  is  not  alarming.  This 
result  was  expected,  as  described  in  Section  3.6.  However,  looking  at  the  percent  of  NMACs 
with  reversed  RAs  for  the  planned  crossing  classes  three  (2,  5,  and  9)  have  more  NMACs  with 
reversed  RAs  than  6.04a;  four  classes  (0,  1,  3,  and  4)  have  no  reversed  RA  NMACs;  and  only 
two  classes  (6  and  7)  have  fewer  NMACs  with  reversed  RAs  compared  to  6.04a.  For  the 
planned  non-crossing  classes  with  Version  7  Mod  11  five  classes  (10,  11,  12,  13,  and  14)  have 
no  reversed  RA  NMACs;  four  classes  (15,  16,  18,  19)  have  more  reversed  RA  NMACs  than 
6.04a;  and  only  one  class  (17)  has  less  reversed  RA  NMACs  with  Version  7  Mod  11  than  with 

6.04a! 

The  European  airspace  currently  has  more  TCAS— unequipped  encounters  than  the  United  States 
airspace.  Our  European  colleagues  expected  to  see  an  increase  in  the  number  of  reversed  sense 
RAs  in  their  simulated  encounters.  Their  conclusion  was  that  the  other  benefits  of  Change  7  far 
outweigh  the  increase  in  reversed  RA  NMACs  in  the  TCAS— unequipped  encounters.  Looking  at 
the  results  from  the  WJHTC  data  Appendix  E  NMAC  Tables,  for  all  twenty  encounter  classes 
the  total  number  of  NMACs  for  6.04a  vs.  Mode  C  is  3360  (783  unresolved  and  2577  induced), 
the  number  of  Change  7-100  vs.  Mode  C  NMACs  is  2586  (783  unresolved  and  1803  induced) 
and  the  number  of  Change  7-25  vs.  Mode  C  NMACs  is  1580  (384  unresolved  and  1196 
induced).  The  Version  7  Mod  1 1  100-foot  tracker  data  show  a  23%  reduction  in  NMACs  against 
xmequipped  threats  compared  to  6.04a  and  the  V ersion  7  Mod  1 1  25-foot  tracker  logic  shows  a 
53%  reduction  in  NMACs  against  unequipped  threats.  This  supports  the  conclusion  of  our 
European  colleagues. 

4.2.4.2  Variation  Two  -  Separation  Differences 

The  results  of  the  Separation  Differences  program  (Reversal  Analysis  Program  Variation  Two) 
fQj-  Version  7  Mod  11  are  quite  encouraging.  The  complete  set  of  results  is  in  Appendix  D.  The 
left  half  of  Figure  4-6  contains  a  summary  of  the  Separation  Difference  results  for  Version  7 
Mod  10,  and  the  right  half  of  Figure  4-6  contains  a  summary  of  the  Separation  Differences 
results  for  Version  7  Mod  11. 

At  first  glance.  Version  7  Mod  11  classes  2/12  had  higher  separation  losses  than  we  expected 
(75.3%)  for  reversed  RAs.  However,  we  are  not  overly  concerned  for  several  reasons.  First, 
Version  7  Mod  10  had  100%  separation  losses,  so  Mod  11  is  an  improvement.  Second,  in 
looking  at  the  gains  and  losses  tables,  all  the  losses  occurred  in  encounters  with  750  feet  of 
planned  separation  and  most  of  the  losses  were  less  that  500  feet.  In  addition,  for  non-reversed 
RAs  in  the  same  encounter  class  the  750  feet  planned  separation  encounters  had  comparable 
losses. 

Most  of  the  other  classes  have  improved  gains.  The  most  dramatic  improvement  occurs  in 
classes  3/13.  Where  100%  of  encounters  lost  separation  with  Version  7  Mod  10,  but  only  4.4% 
of  encounters  lost  separation  with  Version  7  Mod  1 1 . 
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Reversal  Analysis  Program  Results 


Version  7  Mod  10  (May  1997) 

Version  7  M 

od  11  (November  199 

2 - 

Class 

Rev.? 

RAs 

NMACs  %  1 

Sep  Loss 

% 

Sep  Gain 

% 

RAs 

NMAC! 

% 

Sep  Loss 

% 

Sep  Gain 

% 

010 

no 

480 

0 

0.0 

0 

0.0 

336 

70.0 

480 

0 

0.0 

0 

0.0 

336 

70.0 

Rev. 

0 

0 

0.0 

0 

0.0 

0 

0.0 

0 

0 

0.0 

0 

0.0 

0 

0.0 

111 

no 

1800 

0 

0.0 

0 

0.0 

1360 

75.6 

1800 

0 

0.0 

0 

0.0 

1360 

75.6 

Rev. 

0 

0 

0.0 

0 

0.0 

0 

0.0 

0 

0 

0.0 

0 

0.0 

0 

0.0 

212 

No 

22359 

0 

0.0 

1302 

5.8 

16805 

75.2 

22320 

62 

0.3 

1215 

5.4 

16853 

75.5 

Rev. 

34 

0 

0.0 

34 

100.0 

0 

0.0 

73 

2 

2.7 

55 

75.3 

18 

24.7 

313 

No 

19654 

1 

0.0 

34 

0.2 

16104 

81.9 

18777 

0 

0.0 

10 

0.1 

15169 

80.8 

Rev. 

87 

68 

78.2 

87 

100.0 

0 

0.0 

938 

6 

0.6 

41 

4.4 

897 

95.6 

414 

no 

3596 

0 

0.0 

6 

0.2 

3038 

84.5 

3596 

0 

0.0 

6 

0.2 

3038 

84.5 

Rev. 

0 

0 

0.0 

0 

0,0 

0 

0.0 

0 

0 

0.0 

0 

0.0 

0 

0.0 

515 

no 

43413 

78 

0.2 

1513 

3.5 

36414 

83.9 

43237 

30 

0.1 

1329 

3.1 

36422 

84.2 

Rev. 

79 

5 

6.3 

57 

72.2 

22 

27.8 

255 

0 

0.0 

11 

4.3 

244 

95.7 

616 

no 

44560 

82 

0.2 

960 

2.2 

37338 

83.8 

43688 

46 

0.1 

892 

ZO 

36754 

83.7 

Rev. 

46 

5 

10.9 

37 

80.4 

9 

19.6 

737 

8 

1.1 

24 

3.3 

713 

96.7 

717 

no 

65937 

310 

0.5 

5317 

8.1 

53052 

80.5 

65853 

220 

0.3 

5028 

7.6 

53255 

80.9 

Rev. 

43 

6 

14.0 

20 

46.5 

23 

53.5 

127 

2 

1.6 

44 

34.6 

83 

65.4 

81 B 

no 

66150 

412 

0.6 

3581 

5.4 

53321 

80.6 

64672 

333 

0.5 

2957 

4.6 

52518 

81.2 

Rev. 

360 

192 

53.3 

352 

97.8 

8 

2.2 

1899 

19 

1.0 

381 

20.1 

1518 

79.9 

919 

no 

43763 

43 

0.1 

831 

1.9 

38509 

88.0 

42350 

14 

0.0 

687 

1.6 

37216 

87.9 

Rev. 

426 

65 

15.3 

372 

87.3 

54 

12.7 

1843 

34 

1.8 

214 

11.6 

1629 

88.4 

Total 

Non 

311712 

926 

0.3 

13544 

4.3 

256277 

82.2 

306973 

705 

0.2 

12124 

3.9 

252921 

82.4 

Rev 

1075 

341 

31.7 

959 

89.2 

116 

10.8 

5872 

71 

1.2 

770 

13.1 

5102 

86.9 

312787 

1267 

312845 

776 

0.3%  of  RAs  are  reversals 

Figure  4-6.  Comparison  of  Version  7 


1 .9%  of  RAs  are  reversals 

Mod  10  and  Version  7  Mod  11. 


4.3  RESULTS  -  ONE  PILOT  NON-RESPONDING 

As  mentioned  in  Section  1.1.2,  the  data  collected  with  one  pilot  not  responding  to  the  TCAS 
advisory  were  collected  for  6.04a  and  Version  7  Mod  1 1  with  the  25-foot  tracker  only.  When 
one  pilot  does  not  comply  with  the  advisory  the  results  have  always  been  poor.  In  fact,  one  of 
the  most  compelling  reasons  for  allowing  TCAS-TCAS  sense  reversals  was  to  allow  the 
compliant  aircraft  some  means  of  escape  when  involved  in  an  encounter  with  a  non-compliant 
aircraft. 

The  presence  of  an  unresolved  NMAC  when  one  pilot  does  not  follow  the  RA  is  not  overly 
troubling,  due  to  the  geometry  of  the  encounter  there  was  going  to  be  an  NMAC  without  TCAS. 
On  the  other  hand,  the  presence  of  an  induced  NMAC  when  one  pilot  does  not  follow  the  RA  is 
very  disturbing.  In  this  case,  without  TCAS  there  would  have  been  no  NMAC. 
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The  only  analysis  tools  used  on  the  non-responding  data  set  were  the  WJHTC  Matrix  Generator 
program  and  Lincoln  Laboratory’s  Hot-Spot  Pro^am.  The  complete  set  of  one  pilot  non¬ 
responding”  hot-spot  tables  are  provided  in  Appendix  L. 

A  summary  of  the  hot-spot  results  for  both  unresolved  and  induced  NMACs  for  all  twenty 
encounter  classes,  including  the  mixed  6.04a  and  C7-25  equipages,  is  provided  in  Table  4-5. 
Looking  at  Table  4-5,  any  cell  with  degraded  performance,  compared  to  6.04a,  is  indicated  by  a 
bold  border.  Overall,  the  "one  pilot  non-responding"  results  are  quite  good. 


Table  4-5.  Summary  of  Non-Responding  Hot-Spot  Data 
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Figures  4-7  and  4-8  show  histograms  of  the  hot-spot  data  for  6.04a  -  one  pilot  non-responding 
and  C7-25  one  pilot  non-responding.  Figure  4-7  shows  induced  NMACs  and  Figure  4-8  shows 
unresolved  NMACs. 

Most  of  the  United  States  encounters  are  in  classes  10-19,  there  are  very  few  planned  crossing 
encounters.  Our  biggest  concern  is  the  induced  NMACs  in  the  one  pilot  non-responding  data. 
The  most  significant  improvement  was  observed  in  the  induced,  planned  non-crossing 
encounters. 


Encounter  Class 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


6.04a 

C7-25ft 


Encounter  Class 

Figure  4-7.  Hot-Spot  Results,  One  Pilot  Non-Responding  Induced  NMACs. 
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KMACs  M  W  of  PUukcl  Eueounttrs  NMACs  *s  fS  of  PUiuud  Encounters 


6.04a 

C7-25ft 


6.04a 

C7-25ft 


4.4  TCAS  VS.  UNEQUIPPED  ENCOUNTERS 

The  main  focus  of  the  Change  7  logic  evaluation  at  Lincoln  Laboratory  was  the  TCAS-TCAS 
encounters.  However,  there  is  simulated  data  for  TCAS-unequipped  encounters.  From  the 
NMAC  Tables  in  Appendix  E,  we  can  compute  the  total  NMAC  counts  for  TCAS-unequipped 
encounters  There  was  an  overall  reduction  in  NMAC  counts  from  3360  for  6.04a  against 
unequipped  intruders  (783  unresolved,  2577  induced)  to  2586  for  Change  7-100  (783  unresolved, 
1803  induced)  and  1580  for  Change  7-25  (384  unresolved,  1196  induced).  However  the 
performance  of  sense  reversals  against  unequipped  intruders  is  mixed.  Ten  classes  had  no 
NMACs  with  reversed  RAs  for  all  three  versions  of  the  logic.  Six  classes  had  more  NMACs  tor 
Change  7  than  6.04a.  Four  classes  had  fewer  NMACs  for  Change  7  than  6.04a. 
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5.  RESOLUTION  OF  6.04A  REPRESENTATIVE  NMACS 


5.1  DESCRIPTION 

During  the  6.04a  logic  evaluation,  Lincoln  Laboratory  identified  thirty  distinct  groups  of 
NMACs.  These  groups  were  defined  based  on  their  encounter  class,  the  type  of  NMAC  (induced 
or  unresolved),  the  shape  of  the  encounter  plot,  and  the  RAs  generated  by  the  two  aircraft. 
Within  each  group  of  NMACs,  one  encounter  that  best  represented  the  group  was  selected  and 
studied  in  detail.  At  the  time  of  the  6.04a  analysis  the  TCAS  community  expected  that  Change  7 
would  improve  the  outcome  of  most  of  these  Representative  NMAC  encounters.  As  mentioned 
in  Section  1.2,  the  third  goal  of  the  Lincoln  Laboratory  Change  7  logic  analysis  was  to  examine 
the  performance  of  Change  7  for  these  thirty  Representative  NMAC  encounters. 

5.2  RESULTS 

All  thirty  6.04a  representative  NMAC  encounters  were  run  using  versions  6.04a,  Change  7-100 
and  Change  7-25.  Table  5.1  shows  identifying  information  for  each  of  the  6.04a  Representative 
NMAC  encounters  and  the  vertical  separation  achieved  by  each  version  of  the  logic.  An  R  in  the 
left  side  of  the  Change  7  achieved  Separation  column  indicates  that  there  was  a  sense  reversal 
during  the  sequence  of  RAs  observed  for  the  encounter. 

All  seven  unresolved  6.04a  representative  NMACs  (1,  6,  7,  9,  12,  13,  and  14)  were  resolved  by 
the  Change  7  logic. 

Most  of  the  twenty-three  induced  6.04a  representative  NMAC  encounters  showed  the  expected 
improvement  with  a  few  exceptions.  Representative  NMAC  2  was  an  NMAC  for  both 
Change  7-100  and  Change  7-25.  Figure  5-1  shows  the  encounter  summary  for  6.04a 
Representative  NMAC  2.  In  this  encounter,  aircraft  1  is  level,  aircraft  2  is  level,  then  accelerates 
at  0.25g  to  5000  fpm  20  seconds  before  CPA.  This  type  of  encounter,  known  as  a  "kamikaze 
maneuver,"  has  been  troublesome  for  TCAS  since  version  6.04  when  the  protection  volume 
about  the  aircraft  was  reduced. 

Representative  NMAC  15  was  no  longer  an  NMAC  for  Change  7-100,  but  Change  7-25  still 
induced  an  NMAC.  Figure  5-2  shows  the  encounter  summary  for  6.04a  Representative 
NMAC  15.  In  this  encounter,  aircraft  1  is  descending  at  1000  fpm,  then  levels  off,  aircraft  2  is 
level  then  accelerates  at  0.25g  to  5000  fpm  20  seconds  before  CPA.  The  RAs  chosen  by  the  3 
versions  of  the  logic  are  interesting.  Aircraft  2  maneuvered  5  seconds  before  aircraft  1,  so  the 
commands  selected  by  aircraft  2  are  critical.  Version  6.04a  had  a  three  second  display  deferral 
for  mrcraft  2  at  44  seconds  that  contributed  to  the  poor  outcome.  Change  7-100,  which  did  not 
have  an  NMAC  for  this  encounter,  had  a  "descend"  command  at  44  seconds.  Change  7-25, 
which  did  have  an  NMAC,  had  a  "don’t  climb"  command  at  46  seconds,  followed  by  a 
"descend"  command  at  47  seconds. 

Finally,  Representative  NMAC  26  showed  no  improvement  for  Change  7-100  or  Change  7-25. 
Figure  5-3  shows  the  encounter  summary  for  6.04a  Representative  NMAC  26.  In  this  encounter 
both  aircraft  are  initially  level.  Aircraft  1  accelerates  at  0.15  g  to  5000  fpm  at  25  seconds  before 
CPA.  Aircraft  2  accelerates  at  0.1 5g  to  3000  fpm  at  30  seconds  before  CPA.  In  all  three 
versions  of  the  logic,  aircraft  1  with  the  higher  rate  of  climb,  gets  the  "descend"  command. 
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Table  5-1.  Results  for  6.04a  Representative  NMACs  Run  with  Version  7  Mod  11  Logic 


Total  639 

R  indicates  a  sense  reversai  occurred 


These  results  are  very  encouraging,  however  there  are  variations  in  the  timing  of  vertical 
maneuvers  within  a  group  of  NMACs.  It  is  possible  that  while  the  encounter  chosen  as 
representative  of  the  other  NMACs  may  have  a  successful  outcome  with  Change  7,  other 
members  of  the  group  may  still  be  NMACs. 
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DATA  FOR  RNMAC02 


1196  6 

.04A  RL  VS  6.04A  RH  2  -48. 

15  CROSSING_ENCOUNTER 

SL  =  4 

ZTHR  =  600.0  TAUR  =  20.0 

TAUV  = 

20.0 

ALIM  = 

300.0 

-500.0 

(0. 0,0.0)  (0.0,5000.0)  0.00 

0.25 

0.0 

-20.0 

3720.0 

A/ Cl: 

1165022  TA  TIME  :30  | RELZ 

1  CL 

@48 

[NXRA]  1 

ICL  @50 

A/C2: 

1265122  TA  TIME  :30  |RELZ 
[NXRA]  1  IDES  @53 

i  POTRA 

@46 

(DFD)  1 

DES  @48 

1196  Cl  100  FT  RL  VS  C7  100  FT  RH  2 

81.99 

NON_ 

_CROSSING_ 

^ENCOUNTER 

SL  =  4 

ZTHR  =  600,0  TAUR  =  20,0 

TAUV  = 

20.0 

ALIM  = 

300.0 

-500.0 

(0.0, 0.0)  (0.0,5000,0)  0.00 

0.25 

0.0 

-20.0 

3720.0 

A/ Cl: 

1171033  TA  TIME  :30  |RELZ 

1  CL 

@47 

[NXRA]  1 

ICL  @50 

A/C2: 

1271133  TA  TIME  :30  1 RELZ 

1  DCL 

@46 

[NXRA]  1 

DES  @47 

1196  C7  25  FT  RL  VS  Cl  25  ET  RH  2 

56.62 

CROSSING_ENCOUNTER 

SL  =  4 

ZTHR  =  600.0  TAUR  =  20.0 

TAUV  = 

20.0 

ALIM  = 

300.0 

-500.0 

(0. 0,0.0)  (0.0,5000.0)  0.00 

0.25 

0.0 

-20.0 

3720,0 

A/Cl: 

1175044  TA  TIME  :30  )TAUV 
[NXRA] 1  ICL  @53 

1  POTRA 

@46 

(VTT)  I 

CL  047 

A/C2: 

1275144  TA  TIME  :30  |TAUV 

1  DCL 

@46 

[NXRA]  1 

DES  @47 

Figure  5-1.  6.04a  Representative  NMAC  2. 
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DATA  FOR  RNMAC15 

1385  6.04A  RL  VS  6.04A  RH  8  39.36  CROSS ING__ENCOUNTER 

SL  =  4  ZTHR  -  600.0  TAUR  -  20.0  TAUV  =  20.0  ALIM  =  300.0 

-500.0  (-1000.0^0.0)  (0.0,5000.0)  0.05  0.25  -25.0  -20.0  3700.0 

A/Cl:  1165022  TA  TIME  :32  |RELZ  |  CL  @47  [NXRA]  1  ICL  @49 

A/C2:  1265122  TA  TIME  :34  ITAUV  |  POTRA  @44  (DFD)  |  DES  @47 

[NXRA]  I  IDES  @54 


1385  C7  100FTRLVSC7  100FTRH  8  396.02  NON_CROSSING_ENCOUNTER 

SL  =  4  ZTHR  =  600.0  TAUR  =  20.0  TAUV  =  20.0  ALIM  =  300.0 


-500.0 

(-1000.0,0.0) 

(0.0,5000. 

0)  0.05 

0.25  -25.0  -20. 

.0 

3700.0 

A/ Cl: 

1171033 

TA  TIME 

:32 

IRELZ  1 

CL  047  [NXRA]  1 

(  ICL 

@50 

A/C2: 

1271133 

TA  TIME 

;34 

ITAUV  1 

DES  044  [NXRA]  1 

1  IDES 

@53 

I  DCL  @63 


1385  Cl  25  FT  RL  VS  C7  25  FT  RH  8  62. IB  CROSSING_ENCOUNTER 

SL  =  4  ZTHR  =  600.0  TAUR  =  20.0  TAUV  =  20.0  ALIM  =  300.0 


-500.0 

(-1000.0,0.0) 

(0.0,5000.0)  0.05 

0.25  -25.0  -20.0 

3700.0 

A/ Cl: 

1175044 

TA  TIME 

:33  IRELZ  | 

CL  @47  [NXRA]  1  ICL 

@52 

A/C2: 

1275144 

TA  TIME 

:34  IRELZ  1 

DCL  @46  [NXRA]  |  DES 

@47 

Figure  5-2.  6.04a  Representative  NMAC  15. 
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Figure  5-3.  6.04a  Representative  NMAC  26. 
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6.  CHANGE  7  REPRESENTATIVE  NMACS 


The  methodology  of  choosing  representative  NMACs  was  beneficial  in  the  6.04a  logic 
evaluation,  so  the  same  technique  was  applied  in  the  Change  7  evaluation.  Every  encounter  that 
generated  an  NMAC  for  either  Change  7-100  or  Change  7-25  was  plotted  for  all  three  versions 
of  the  logic  (6.04a,  Change  7-100,  and  Change7-25).  These  groups  of  plots  were  sorted  first  by 
encounter  class  and  NMAC  type  (induced  or  unresolved),  and  then  they  were  sorted  by  the  shape 
of  the  encounter  plots  and  the  sense  of  the  RAs  generated  by  the  two  TCAS  aircraft.  Out  of 
twenty  encounter  classes  there  were  only  seventeen  different  categories  of  NMACs,  three  groups 
of  unresolved  NMACs  and  14  groups  of  induced  NMACs. 

Keeping  in  mind  that  the  WJHTC  simulation  is  designed  to  stress  the  limits  of  the  logic,  for  each 
of  the  17  representative  NMACs,  we  attempted  to  answer  the  following  questions: 

1.  Did  we  understand  the  NMAC?  Do  we  know  the  cause  of  the  NMAC?  Was  there 
anything  about  the  performance  of  the  logic  that  seemed  wrong?  In  other  words  are 
these  NMACs  the  types  of  encounters  that  TCAS  cannot  reasonably  be  expected  to 
resolve? 

2.  How  frequently  will  the  encounter  be  observed  in  the  airspace?  Would  the  encounter 
geometry  occur  during  normal  operations  or  only  during  some  breakdown  or  error  in 
the  air  traffic  control  system?  In  other  words  are  these  NMACs  unlikely  to  occur  in  the 
airspace? 

3 .  Is  this  NMAC  type  new  to  Change  7  or  similar  to  NMACs  observed  in  6.04a? 

Only  one  Representative  NMAC  group  was  new  to  Change  7  (see  Representetive  NMAC  114 
below).  All  seventeen  Representative  NMACs  for  Change  7  fell  into  the  "unlikely  to  occur"  or 
"impossible  to  fix"  categories.  Each  of  the  17  Representative  NMACs  are  described  below  in 
Table  6-1. 

Table  6-1  gives  a  summary  of  the  common  features  of  each  NMAC  group  including;  the  Matrix 
table  from  which  the  group  of  encounters  came;  the  number  of  NMACs  in  the  ^oup;  the  planned 
separation  of  the  encounters  in  the  group;  the  presence  of  crossing  RAs  in  the  group;  the 
presence  of  geometric  reversals  for  most  encounters  in  the  group;  the  vertical  rates  observed  for 
aircraft  1  and  aircraft  2  in  the  group;  and  the  comparable  6.04a  NMAC  group  number,  if 
applicable. 
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Table  6-1.  Change  7  Representative  NMAC  Encounters 


UNRESOLVED  NMACs  (Total  22) 


Group 

(table) 

No.  Of 

NMACs 

Planned 

Separation 

Crossing 

RA? 

Geom 

Rev? 

Aircraft  Rates 

6.04a 

NMAC 

group 

AC1 

AC2 

1.  (6.3-1) 

8 

0 

Non-cross 

No 

5000 

5000 

7 

2.  (6.3-2) 

4 

0 

Crossing 

Yes 

>=3000 

<=-3000 

7 

3.  (8.3-1) 

10 

0 

Non-cross 

No 

<=-3000 

-5000 

12 

INDUCED  NMACs  (Total  385) 


Group 

(table) 

No.  Of 

NMACs 

Planned 

Separation 

Crossing 

RA? 

Geom 

Rev? 

Aircraft  Rates 

6.04a 

NMAC 

group 

AC1 

AC2 

1.  (2.4-1) 

32 

-500, -750 

Non-cross 

No* 

0,  +/-400 

5000 

2 

2.  (5.4-1) 

1 

500 

Non-cross 

No 

5000 

3000 

4 

3.  (5.4-2) 

10 

-750 

Non-cross 

No 

-1000 

5000 

5 

4.  (6.4-1) 

16 

-500 

Non-cross 

No 

-5000 

-5000 

8 

5.  (7.4-1) 

1 

500 

Non-cross 

No 

5000 

3000 

26 

6.  (7.4-2) 

118 

250  -  750 

Non-cross 

No 

>=  1000 

<=-1000 

9 

7.  (8.4-1) 

24 

-500 

Non-cross 

No* 

<=  -1000 

-5000 

17 

8.  (8.4-2) 

130 

250  -  750 

Non-cross 

No* 

<=-1000 

<=  -3000 

16 

9.  (9.4-1) 

7 

-250 

Crossing 

No* 

5000 

-5000 

19 

10.  (13.4-1) 

3 

250 

Crossing 

Yes 

0,400 

5000 

22 

11.(15.4-1) 

4 

-500 

Crossing 

No 

5000 

5000 

23 

12.  (17.4-1) 

13 

500 

Crossing 

No* 

5000 

3000 

26 

13.  (18.4-1) 

10 

750 

Crossing 

Yes 

-5000 

<=  -1000 

27 

14.  (19.4-1) 

16 

-250, -500 

Crossing 

Yes 

>=  1000 

<=  -3000 

none 

*  Means  a  small  number  of  encounters  had  reversals  that  came  too  late  to  be  effective. 


Appendix  N  contains  five  pages  that  describe  each  Change  7  Representative  NMAC.  The  first 
page  gives  the  Encounter  class  and  unique  encounter  number  of  the  representative  NMAC,  then 
provides  statistics  on  the  parameters  of  the  NMAC  group  as  a  whole.  In  addition,  relevant 
performance  statistics  for  the  whole  encounter  class  are  listed,  and  comments  are  provided.  The 
second  page  is  the  Encounter  Summary  for  the  Representative  NMAC.  These  were  descnbed 
previously  in  Section  2.5.  The  third  page  is  a  plot  of  the  Representative  NMAC  encounter,  but 
with  version  6.04a  for  reference  purposes.  The  fourth  page  is  a  plot  of  the  Representative 
NMAC  encounter  with  the  Change  7  100-ft  tracker  logic.  The  fifth  page  is  a  plot  of  the 
Representative  NMAC  encounter  with  the  Change  7  25-ft  tracker  logic.  The  unresolved 
Representative  NMACs  are  indicated  with  UNR  or  U  on  the  top  line  of  each  page.  Induced 
Representative  NMACs  are  indicated  by  IND  or  I  on  the  top  line  of  the  page. 
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6.1  UNRESOLVED  REPRESENTATIVE  NMACS 


Representative  NMAC  U1  has  eight  members  in  the  group.  For  these  NMACs,  the  RAs  came 
too  late  to  be  effective. 

Representative  NMAC  U2  has  four  members  in  the  group.  All  encounters  in  this  group  had 
sense  reversals  that  occurred  late  in  the  encounter. 

Representative  NMAC  U3  has  ten  members  in  the  group.  All  of  these  encounters  had  late 
maneuvers  at  high  rates. 

All  of  the  unresolved  NMACs  in  these  three  groups  had  vertical  rates  for  both  aircraft  of 
3000  fpm  or  higher. 

6.2  INDUCED  UNRESOLVED  NMACS 

Representative  NMAC  II  has  32  encounters  with  aircraft  1  level  or  nearly  level  (+/-  400  fpm), 
but  aircraft  2  accelerating  at  .25  g  or  .35  g  20  seconds  prior  to  CPA.  This  type  of  encounter  was 
also  a  problem  for  the  6.04a  logic. 

Representative  NMAC  12  is  a  group  with  only  one  member.  In  this  case  the  Change  7-25  logic 
picked  a  different  sense  than  6.04a  or  Change  7-100  which  were  non-NMACs. 

Representative  NMAC  13  has  ten  members  in  the  group.  These  encounters  all  had  late 
maneuvers  at  high  rates.  In  the  example  shown  on  the  plots,  the  6.04a  logic  picked  the  same 
commands  as  Change  7-100  and  Change  7-25.  The  only  difference  is  that  6.04a  issued  the  initial 
command  or  the  increase  command  a  second  earlier  than  Change  7. 

Representative  NMAC  14  has  16  members  in  the  group.  All  of  these  encounters  had  late 
maneuvers  at  high  rates  by  aircraft  2. 

Representative  NMAC  15  is  another  group  with  only  one  member.  Again  the  Change  7-25  logic 
picked  a  different  sense  than  6.04a  or  Change  7-100  which  were  non-NMACs. 

Representative  NMAC  16  has  118  members  in  the  group.  The  NMACs  in  this  group  cover  all 
vertical  rates  used  in  the  simulation  (1000,  3000  and  5000  ^m  climb  or  descend),  three  different 
planned  separations  (250,  500,  and  750  feet),  and  several  rates  of  acceleration.  The  common 
mechanism  in  the  NMAC  is  that  the  two  aircraft  were  intending  to  cross  in  altitude  and  TCAS 
issued  non-crossing  RAs. 

Representative  NMAC  17  has  24  members  in  the  group.  In  this  case,  aircraft  2  climbed  through 
aircraft  I’s  altitude,  the  two  aircraft  wanted  to  cross  and  TCAS  issued  non-crossing  RAs.  This 
type  of  encounter  was  also  a  trouble  spot  for  6.04a. 

Representative  NMAC  18  is  the  largest  group  with  130  members.  Again  several  planned 
separations  and  several  combinations  of  vertical  rates  and  accelerations  are  present.  The  reason 
these  encounters  failed  is  that  the  aircraft  planned  to  cross  in  altitude  and  TCAS  issued  non¬ 
crossing  RAs. 

Representative  NMAC  19  has  seven  members.  These  encounters  are  high  rate  level-offs  that  fail 
because  the  logic  delays  issuing  a  crossing  RA  while  waiting  for  the  level-off.  This  type  of 
encounter  was  also  a  trouble  spot  for  6.04a. 
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Representative  NMAC  110  has  three  members  in  the  group.  These  encounters  have  aircraft  1 
level  or  nearly  level  with  aircraft  2  rapidly  changing  from  5000  fpm  to  level  flight.  All  of  the 
failures  had  250  feet  of  planned  separation  and  all  had  the  100-foot  tracker.  More  significantly, 
all  encounters  had  sense  reversals.  The  6.04a  version  of  the  logic  issued  commands  that  crossed 
the  aircraft  in  altitude.  Change  7  picked  the  same  initial  sense  commands  as  6.04a  but  reversed 

sense  and  barely  failed. 

Representative  NMAC  Ill  has  four  members  in  the  group.  All  of  these  encounters  had  late 
maneuvers  at  high  rates  by  aircraft  2. 

Representative  NMAC  112  has  13  members  in  the  group.  Both  aircraft  were  level  then  began 
accelerating  to  5000  fpm  at  the  same  time.  This  type  of  encounter  was  also  a  problem  for  6.04a. 

Representative  NMAC  113  has  ten  members  in  the  group,  9  occurred  with  the  100-foot  tracker. 
These  encounters  are  somewhat  troubling  because  the  aircraft  did  not  intend  to  cross  in  altitude, 
a  command  was  selected  that  was  contrary  to  the  planned  aircraft  motion,  the  command  was 
reversed,  but  too  late  to  correct  the  problem.  In  this  case,  a  strengthening  (as  in  the  6.04a  plot) 
would  have  lead  to  a  better  conclusion  than  the  sense  reversal.  The  best  outcome  is  obtained 
when  sense  of  the  command  matches  the  planned  aircraft  motion,  as  in  the  Change  7-25  plot. 

Finally,  Representative  NMAC  114  has  16  members,  most  (81%)  occurred  with  the  100-foot 
tracker.  There  is  no  comparable  representative  NMAC  group  from  6.04a.  Again,  this  is  an 
encounter  where  the  two  aircraft  do  not  intend  to  cross  in  altitude,  a  command  was  selected  that 
was  contrary  to  the  planned  aircraft  motion,  then  command  was  reversed,  but  too  late  to  correct 

the  problem. 
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7.  PERSPECTIVE 


7.1  CHRONOLOGY  OF  REVISIONS 

A  thorough  study  of  the  proposed  Change  7  logic  was  carried  out  at  Lincoln  Laboratory.  During 
the  evaluation,  Lincoln  Laboratory  focused  on  the  TCAS— TCAS  reversals  section  of  the  logic. 
Lincoln  Laboratory  detected  problems  with  the  reversal  logic  (the  occurrence  of  multiple 
reversals)  that  were  corrected  by  revisions  in  the  reversal  logic. 

An  FAA-sponsored  assessment  team  (including  MITRE,  WJHTC,  and  Lincoln  Laboratory)  then 
undertook  an  effort  to  decide  if  reversals  should  be  included  in  the  Change  7  logic.  Lincoln 
Laboratory  pointed  out  the  relatively  poor  performance  of  TCAS— TCAS  reversals  in  class  1 3 
(the  "Dallas  bump  up"  class)  and  the  marginal  improvement  in  only  five  of  the  twenty  encounter 
classes  for  a  non-responding  intruder  in  TCAS-TCAS  coordinated  encounters.  Lincoln 
Laboratory  also  pointed  out  that  the  overall  performance  for  both  pilots  responding  was  worse 
with  reversals  than  without  reversals.  Based  on  the  MITRE  safety  study  results  that  showed 
overall  satisfactory  performance,  and  the  lack  of  overwhelming  evidence  for  the  removal  of 
TCAS-TCAS  reversals,  the  assessment  team  decided  to  proceed  with  the  implementation  of 
Change  7  with  TCAS-TCAS  reversals. 

Lincoln  Laboratory  then  studied  the  behavior  of  all  reversed  RA  encounters  and  non-reversed 
RA  encounters.  The  study  showed  that  reversed  RA  encounters  had  an  overwhelming  tendency 
to  lose  separation,  while  non-reversed  RA  encounters  predominantly  increased  separation. 
These  findings  led  to  another  fix  of  the  logic.  The  performance  of  TCAS-TCAS  reversals  then 
improved  dramatically.  With  this  improvement,  the  assessment  team  deemed  the  Change  7  logic 
to  be  suitable  for  implementation. 

7.2  PERFORMANCE  TRENDS 

The  focus  of  the  Change  7  analysis  was  to  compare  the  performance  of  the  Change  7  logic  to 
version  6.04a.  There  were  two  previously  implemented  versions  of  the  TCAS  logic,  namely  6.02 
and  6.04.  In  order  to  gain  some  perspective  on  the  evolution  of  TCAS,  the  behavior  of  all  five 
versions  of  TCAS  logic  is  summarized  in  three  tables.  Table  7.1  shows  the  number  of 
unresolved  NMACs  by  encounter  class  and  logic  version,  expressed  as  a  percentage  of  the 
number  of  planned  encounters.  Table  7.2  shows  the  number  of  induced  NMACs  by  encounter 
class  and  logic  version,  expressed  as  a  percentage  of  the  number  of  planned  encounters.  Finally, 
Table  7.3  shows  the  total  number  of  NMACs  by  logic  version. 

In  Table  7-1,  note  there  were  no  unresolved  encounters  for  logic  version  6.02.  For  6.04,  the 
protection  volume  was  reduced  in  size  to  mitigate  the  nuisance  RAs  characteristic  of  6.02.  This 
introduced  unresolved  NMACs.  Since  version  6.04,  with  the  exception  of  encounter  class  6, 
there  has  been  a  distinct  reduction  in  unresolved  NMACs. 

In  Table  7-2,  note  that  from  version  6.02  to  version  6.04,  there  was  an  increase  in  induced 
NMACs  in  eveiy  encounter  class.  The  general  trend  from  6.04  to  6.04a  to  Change  7  is  a 
reduction  in  induced  NMAC  counts.  The  most  dramatic  example  of  improvement  is  observed  in 
encounter  class  19. 

In  Table  7-3,  note  that  with  Change  7,  the  total  NMACs  observed  are  below  the  totals  for  6.02. 
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Table  7-1.  Unresolved  NMAC  History 


Class 

6.02 

6.04 

6.04a 

C7-100 

C7-25 

0 

o 

6 

0.0 

0.0 

O 

6 

0.0 

1 

0.0 

0.0 

0.0 

0.0 

0.0 

2 

0.0 

0.579 

0.579 

0.0 

0.0 

3 

0.0 

0.0 

0.0 

0.0 

0.0 

4 

0.0 

0.0 

0.0 

0.0 

0.0 

5 

0.0 

0.0 

0.0 

0.0 

0.0 

6 

0.0 

0.289 

0.810 

0.347 

0.347 

7 

0.0 

0.166 

0.165 

0.0 

0.0 

8 

0.0 

1.157 

1.157 

0.231 

0.154 

9 

0.0 

0.173 

0.0 

0.0 

0.0 

10 

0.0 

0.0 

0.0 

0.0 

0.0 

11 

0.0 

0.0 

0.0 

0.0 

0.0 

12 

0.0 

0.0 

0.0 

0.0 

0.0 

13 

0.0 

0.0 

0.0 

0.0 

0.0 

14 

0.0 

0.0 

0.0 

0.0 

0.0 

15 

0.0 

0.0 

0.0 

0.0 

0.0 

16 

0.0 

0.0 

0.0 

0.0 

0.0 

17 

0.0 

0.0 

0.0 

0.0 

0.0 

18 

0.0 

0.0 

0.0 

0.0 

0.0 

19 

0.0 

0.0 

0.0 

0.0 

0.0 
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Table  7-2.  Induced  NMAC  History 


Class 

6.02 

6.04 

6.04a 

C7-100 

C7-25 

0 

0.0 

0.0 

0.0 

0.0 

0.0 

1 

0.0 

0.0 

0.0 

0.0 

0.0 

2 

0.396 

1.715 

1.707 

0.451 

0.580 

3 

0.0 

0.0 

0.0 

0.0 

0.0 

4 

0.0 

0.0 

0.0 

0.0 

0.0 

5 

0.053 

0.347 

0.367 

0.051 

0.089 

6 

0.379 

0.813 

0.153 

0.153 

0.051 

7 

0.971 

1.912 

1.951 

0.836 

0.348 

8 

0.378 

2.031 

1.649 

0.780 

0.570 

9 

0.017 

0.224 

0.280 

0.078 

0.031 

10 

0.0 

0.0 

0.0 

0.0 

0.0 

11 

0.0 

0.0 

0.0 

0.0 

0.0 

12 

0.0 

0.046 

0.0 

0.0 

0.0 

13 

0.0 

0.467 

0.101 

0.076 

0.0 

14 

0.0 

0.0 

0.0 

0.0 

0.0 

15 

0.0 

0.972 

0.728 

0.081 

0.081 

16 

0.503 

0.771 

0.039 

0.0 

0.0 

17 

0.090 

0.377 

0.254 

0.127 

0.109 

18 

0.254 

0.527 

0.410 

0.217 

0.024 

19 

2.458 

2.907 

0.486 

0.176 

0.041 

Table  7-3.  Total  NMACs  by  Logic  Version 


TCAS  Logic  Version 

Total  NMACs 

Percent 

6.02 

411 

0.449 

6.04 

848 

0.926 

6.04a 

639 

0.698 

C7-100 

254 

0.277 

C7-25 

153 

0.167 
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EPILOGUE 


During  TCAS  certification  testing,  the  logic  is  run  through  detailed  end-to-end  system  tests 
involving  all  components  of  the  system  (e.g.,  TCAS  unit.  Mode  S  transponder)  in  a  deliberate 
attempt  to  expose  the  logic  to  various  kinds  of  surveillance  and  environmental  error  conditions. 
This  process  tests  parts  of  the  logic  that  are  not  covered  by  WJHTC  or  MITRE  testing. 

During  manufacturer  certification  testing  using  the  RTCA-approved  Version  7  Mod  11  logic 
(described  in  the  body  of  this  report),  three  problems  were  identified  that  were  considered  to 
represent  potential  safety  problems.  The  FAA  Certification  Office  indicated  they  would  not 
approve  TCAS  units  for  certification  until  these  problems  were  resolved. 

The  three  problems  were  as  follows: 

1 .  A  problem  with  "pop-up"  targets  was  observed.  Specifically,  there  were  circumstances 
under  which  the  Change  7  CAS  logic  failed  to  issue  an  RA  or  issued  an  RA  later  than 
previous  logic  versions.  This  situation  was  addressed  in  CP  73 

2.  A  problem  was  observed  when  the  Change  7  logic  was  exposed  to  a  faulty 
communications  link  between  two  TCAS-equipped  aircraft.  CP  83  was  proposed  to 
ensure  that  even  if  own  aircraft’s  communication  link  with  the  threat  aircraft  is  faulty 
(own  aircraft  receiving  one  out  of  three  surveillance  replies),  own  aircraft  will  issue  an 
RA  because  the  threat  aircraft  had  previously  sent  its  intent. 

3.  A  problem  was  observed  in  the  multi-aircraft  logic.  In  this  situation  the  logic  failed  to 
change  a  Don’t  Climb/Don’t  Descend  RA  to  a  Descend  RA  when  one  of  the  threat 
aircraft  suddenly  maneuvered  toward  own  aircraft.  This  situation  was  addressed  in 
CP  63. 

MITRE,  Lincoln  Laboratory,  WJHTC  and  our  European  colleagues  at  CENA/DGAC  were 
tasked  to  analyze  the  effectiveness  of  the  three  proposed  solutions,  as  well  as  their  impact  on  the 
overall  Change  7  performance.  Since  these  three  problems  were  being  addressed,  other  less 
critical  Change  Proposals  (CPs)  in  the  queue  were  also  addressed. 

Early  in  November  1998,  WJHTC  provided  Lincoln  Laboratory  with  FTEG  simulated  encounter 
data  for  a  version  of  the  logic  that  incorporated  the  CPs  mentioned  above.  Both  WJHTC  and 
Lincoln  Laboratory  noted  new  reversed  RA  NMACs  that  appeared  in  encounter  classes  where 
we  previously  did  not  observe  any  NMACs. 

Our  CENA/DGAC  ACAS  colleagues,  noting  these  new  NMACs,  ran  some  tests  using  the 
European  model  simulations  with  the  same  logic  as  used  by  WJHTC,  except  with  a  modified  CP 
73.  Their  results  were  promising  enough  to  merit  a  new  collection  of  FTEG  encounters  by 
WJHTC,  also  with  the  modified  CP  73.  This  logic  became  known  as  TSO-C119B,  referring  to 
the  name  of  the  TSO  for  TCAS  Change  7. 

The  Lincoln  Laboratory  logic  evaluation  found  that  the  results  for  TSO-C119B  were  comparable 
to  or  better  than  the  results  for  Version  7  Mod  11.  Specifically,  as  shown  in  Table  7.3  (both 
pilots  responding),  there  were  254  NMACs  for  Change  7  Mod  1 1  with  the  100-foot  tracker  and 
153  NMACs  for  Change  7  Mod  11  with  the  25-foot  tracker.  The  TSO-C119B  Change  7  logic 
had  254  NMACs  with  the  100-foot  tracker  and  1 17  NMACs  with  the  25-foot  tracker.  Likewise, 
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for  one  pilot  not  responding,  the  results  for  TSO-C119B  were  comparable  to  or  better  than  the 
results  for  Version  7  Mod  11. 

Thus  the  TSO-Cl  19B  logic  appeared  to  resolve  the  three  potential  safety  problems  while  at  the 
same  time  improving  the  overall  Change  7  performance.  During  a  teleconference  in  late 
November  1998,  members  of  the  Requirements  Working  Group  of  SC-147  (including  Lincoln 
Laboratory)  unanimously  agreed  to  accept  the  TSO-Cl  19B  logic.  The  changes  required  to 
upgrade  Version  7  Mod  11  to  TSO-Cl  19B  have  been  included  on  a  CD  made  available  to  the 
FAA  Certification  Office  for  use  in  certification  of  manufacturers’  TCAS  Change  7  units. 
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